2024-09-16T06:33:59.579 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T06:33:59.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T06:33:59.680 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576 branch: reef description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906576' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 osd_recovery_max_active: 10 osd_recovery_max_single_start: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.1 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5WngGd5C+n1xBEgR/2fLngmLEmV0kcKYKLwoTCspgnGubGFf9xwLlvHsJDxr2o953DxJKfwcIpulEOuZxr8yE= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGUu56rlMc0r3OgPFOfwbcLd/UNst1IOzCJ9yR7NJX71l+Py0F5QG762RDSs2gZdXCdmvlHgZN9deLQM0rbYEk= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7FDZqLGTRvt5hJym7QalftMihip+TP1QjSm5eydH2p5Vzm/5ac6f9kwBbEhqrOn45zoKAuGR73MOfztco2wzM= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIklEbjPEULrsj2slKPj7JWlq8NAxwUzzeii0j4N99r7c6lEMNVI5oNHF7YYFUdKQBgQwRVXcKM/W216AAwZloU= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: bluestore_new_sharding: random chance_bluestore_reshard: 1 chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T06:33:59.680 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T06:33:59.681 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T06:33:59.681 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T06:33:59.683 INFO:teuthology.task.internal:Checking packages... 2024-09-16T06:33:59.704 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T06:33:59.705 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T06:33:59.705 INFO:teuthology.packaging:ref: None 2024-09-16T06:33:59.705 INFO:teuthology.packaging:tag: None 2024-09-16T06:33:59.705 INFO:teuthology.packaging:branch: reef 2024-09-16T06:33:59.705 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:33:59.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-09-16T06:33:59.973 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853-gae7f3733-1jammy 2024-09-16T06:33:59.975 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T06:33:59.982 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T06:33:59.982 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T06:34:00.021 INFO:teuthology.task.internal:Saving configuration 2024-09-16T06:34:00.036 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T06:34:00.048 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T06:34:00.070 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 06:28:19.457938', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDs07awwR4Dnd933zzKalnESW807hRrCSQ04eIx7XbEU'} 2024-09-16T06:34:00.088 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 06:28:19.461617', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICOymZj3c+zNomdr3D5nE3FszYkOZARj6ExXLlGtWvpN'} 2024-09-16T06:34:00.106 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 06:28:19.463185', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGxWkLLkTEt3uIO/b9OnuNwEuQw4B5ya/ng4n+gx6+pW'} 2024-09-16T06:34:00.123 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 06:28:19.459891', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBa+y1GSdE/spJdW4EAdea1RwurAm7X0l/xYI9olS/TT'} 2024-09-16T06:34:00.124 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T06:34:00.137 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-09-16T06:34:00.137 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-09-16T06:34:00.137 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-09-16T06:34:00.137 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-09-16T06:34:00.137 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T06:34:00.266 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd5325e3010>, signals=[15]) 2024-09-16T06:34:00.266 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T06:34:00.273 INFO:teuthology.task.internal:Opening connections... 2024-09-16T06:34:00.273 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:34:00.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:34:00.356 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:34:00.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:34:00.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:34:00.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:34:00.527 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:34:00.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:34:00.607 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T06:34:00.615 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-16T06:34:00.620 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-16T06:34:00.620 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-16T06:34:00.667 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T06:34:00.667 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2024-09-16T06:34:00.667 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="22.04" 2024-09-16T06:34:00.667 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T06:34:00.667 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=jammy 2024-09-16T06:34:00.667 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2024-09-16T06:34:00.668 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2024-09-16T06:34:00.668 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T06:34:00.668 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T06:34:00.668 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T06:34:00.668 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T06:34:00.668 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=jammy 2024-09-16T06:34:00.668 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-16T06:34:00.706 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-16T06:34:00.712 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-16T06:34:00.712 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-16T06:34:00.758 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T06:34:00.758 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Ubuntu" 2024-09-16T06:34:00.758 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="22.04" 2024-09-16T06:34:00.758 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_CODENAME=jammy 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:ID=ubuntu 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE=debian 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T06:34:00.759 INFO:teuthology.orchestra.run.smithi107.stdout:UBUNTU_CODENAME=jammy 2024-09-16T06:34:00.759 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-16T06:34:00.788 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-16T06:34:00.793 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-16T06:34:00.794 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-16T06:34:00.840 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T06:34:00.840 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Ubuntu" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="22.04" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_CODENAME=jammy 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:ID=ubuntu 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE=debian 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T06:34:00.841 INFO:teuthology.orchestra.run.smithi115.stdout:UBUNTU_CODENAME=jammy 2024-09-16T06:34:00.842 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-16T06:34:00.872 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-16T06:34:00.877 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-16T06:34:00.878 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-16T06:34:00.924 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="22.04" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=jammy 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T06:34:00.925 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=jammy 2024-09-16T06:34:00.926 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-16T06:34:00.946 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T06:34:00.957 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T06:34:00.979 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T06:34:00.980 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-16T06:34:00.983 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-16T06:34:00.985 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-16T06:34:00.988 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-16T06:34:00.992 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T06:34:01.003 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T06:34:01.003 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-16T06:34:01.028 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-16T06:34:01.031 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-16T06:34:01.034 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-16T06:34:01.042 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T06:34:01.130 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T06:34:01.145 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T06:34:01.145 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T06:34:01.145 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T06:34:01.146 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T06:34:01.146 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T06:34:01.147 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T06:34:01.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:34:01.152 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-16T06:34:01.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:34:01.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:34:01.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:34:01.154 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-16T06:34:01.154 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-16T06:34:01.155 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-16T06:34:01.198 INFO:teuthology.orchestra.run.smithi107.stdout:5.15.0-56-generic 2024-09-16T06:34:01.199 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.15.0-56-generic 2024-09-16T06:34:01.199 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get clean 2024-09-16T06:34:01.200 INFO:teuthology.orchestra.run.smithi115.stdout:5.15.0-56-generic 2024-09-16T06:34:01.200 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.15.0-56-generic 2024-09-16T06:34:01.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get clean 2024-09-16T06:34:01.202 INFO:teuthology.orchestra.run.smithi117.stdout:5.15.0-56-generic 2024-09-16T06:34:01.202 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.15.0-56-generic 2024-09-16T06:34:01.202 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2024-09-16T06:34:01.203 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-56-generic 2024-09-16T06:34:01.204 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.15.0-56-generic 2024-09-16T06:34:01.204 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-09-16T06:34:01.328 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2024-09-16T06:34:01.331 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-09-16T06:34:01.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-09-16T06:34:01.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-09-16T06:34:01.448 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:01.464 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:01.465 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:34:01.465 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T06:34:01.494 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T06:34:01.503 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:01.503 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T06:34:01.503 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:34:01.508 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:34:01.541 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T06:34:01.562 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T06:34:01.564 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T06:34:01.565 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:34:01.625 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:01.681 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T06:34:01.686 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T06:34:01.706 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T06:34:01.728 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T06:34:01.737 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T06:34:01.749 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T06:34:01.750 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-16T06:34:01.757 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T06:34:01.783 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-16T06:34:01.786 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T06:34:01.787 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T06:34:01.787 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T06:34:01.788 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-16T06:34:01.790 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T06:34:01.791 INFO:teuthology.orchestra.run.smithi115.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T06:34:01.797 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T06:34:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-16T06:34:01.804 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T06:34:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T06:34:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T06:34:01.814 INFO:teuthology.orchestra.run.smithi107.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T06:34:01.816 INFO:teuthology.orchestra.run.smithi115.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T06:34:01.817 INFO:teuthology.orchestra.run.smithi115.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T06:34:01.817 INFO:teuthology.orchestra.run.smithi115.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-16T06:34:01.819 INFO:teuthology.orchestra.run.smithi107.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T06:34:01.821 INFO:teuthology.orchestra.run.smithi107.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T06:34:01.822 INFO:teuthology.orchestra.run.smithi107.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T06:34:01.822 INFO:teuthology.orchestra.run.smithi115.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T06:34:01.822 INFO:teuthology.orchestra.run.smithi115.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T06:34:01.828 INFO:teuthology.orchestra.run.smithi115.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T06:34:01.832 INFO:teuthology.orchestra.run.smithi115.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T06:34:01.835 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T06:34:01.836 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T06:34:01.837 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T06:34:01.838 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T06:34:01.841 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T06:34:01.851 INFO:teuthology.orchestra.run.smithi107.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T06:34:01.851 INFO:teuthology.orchestra.run.smithi107.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T06:34:01.851 INFO:teuthology.orchestra.run.smithi107.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T06:34:01.851 INFO:teuthology.orchestra.run.smithi107.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T06:34:01.852 INFO:teuthology.orchestra.run.smithi107.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T06:34:01.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T06:34:01.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T06:34:01.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-16T06:34:01.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T06:34:01.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T06:34:01.854 INFO:teuthology.orchestra.run.smithi107.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T06:34:01.854 INFO:teuthology.orchestra.run.smithi107.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T06:34:01.854 INFO:teuthology.orchestra.run.smithi107.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T06:34:01.855 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T06:34:01.856 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T06:34:01.858 INFO:teuthology.orchestra.run.smithi115.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T06:34:01.858 INFO:teuthology.orchestra.run.smithi115.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T06:34:01.858 INFO:teuthology.orchestra.run.smithi115.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-16T06:34:01.865 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-16T06:34:01.867 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T06:34:01.868 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T06:34:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T06:34:01.871 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T06:34:01.871 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T06:34:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T06:34:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T06:34:01.873 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T06:34:01.874 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T06:34:01.874 INFO:teuthology.orchestra.run.smithi115.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T06:34:01.874 INFO:teuthology.orchestra.run.smithi115.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-16T06:34:01.876 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T06:34:01.877 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T06:34:01.877 INFO:teuthology.orchestra.run.smithi115.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T06:34:01.877 INFO:teuthology.orchestra.run.smithi115.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T06:34:01.877 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T06:34:01.881 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T06:34:01.884 INFO:teuthology.orchestra.run.smithi115.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T06:34:01.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T06:34:01.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T06:34:01.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T06:34:01.888 INFO:teuthology.orchestra.run.smithi115.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T06:34:01.890 INFO:teuthology.orchestra.run.smithi115.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T06:34:01.890 INFO:teuthology.orchestra.run.smithi115.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T06:34:01.897 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T06:34:01.901 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-16T06:34:01.901 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T06:34:01.902 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T06:34:01.903 INFO:teuthology.orchestra.run.smithi115.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T06:34:01.907 INFO:teuthology.orchestra.run.smithi115.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T06:34:01.907 INFO:teuthology.orchestra.run.smithi115.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T06:34:01.907 INFO:teuthology.orchestra.run.smithi115.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T06:34:01.907 INFO:teuthology.orchestra.run.smithi115.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T06:34:01.908 INFO:teuthology.orchestra.run.smithi115.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T06:34:01.908 INFO:teuthology.orchestra.run.smithi115.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T06:34:01.908 INFO:teuthology.orchestra.run.smithi115.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T06:34:01.908 INFO:teuthology.orchestra.run.smithi115.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T06:34:01.919 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T06:34:01.919 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T06:34:01.925 INFO:teuthology.orchestra.run.smithi115.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T06:34:01.925 INFO:teuthology.orchestra.run.smithi115.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T06:34:01.936 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T06:34:01.936 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T06:34:01.936 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T06:34:01.937 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T06:34:01.937 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T06:34:01.937 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T06:34:01.937 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-16T06:34:01.953 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-16T06:34:01.957 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T06:34:01.957 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T06:34:01.966 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T06:34:01.973 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T06:34:01.975 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T06:34:01.975 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T06:34:01.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T06:34:01.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T06:34:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T06:34:01.988 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T06:34:01.989 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T06:34:01.990 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T06:34:02.005 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T06:34:02.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T06:34:02.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T06:34:02.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T06:34:02.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T06:34:02.013 INFO:teuthology.orchestra.run.smithi107.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T06:34:02.066 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T06:34:02.206 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T06:34:02.309 INFO:teuthology.orchestra.run.smithi107.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T06:34:02.401 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-16T06:34:02.484 INFO:teuthology.orchestra.run.smithi107.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T06:34:02.496 INFO:teuthology.orchestra.run.smithi107.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T06:34:02.497 INFO:teuthology.orchestra.run.smithi107.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T06:34:02.500 INFO:teuthology.orchestra.run.smithi107.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T06:34:02.561 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-16T06:34:02.575 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T06:34:02.575 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-16T06:34:02.583 INFO:teuthology.orchestra.run.smithi107.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T06:34:02.591 INFO:teuthology.orchestra.run.smithi107.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T06:34:02.591 INFO:teuthology.orchestra.run.smithi107.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T06:34:02.609 INFO:teuthology.orchestra.run.smithi107.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T06:34:02.643 INFO:teuthology.orchestra.run.smithi107.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T06:34:02.645 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T06:34:02.646 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-16T06:34:02.648 INFO:teuthology.orchestra.run.smithi107.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T06:34:02.649 INFO:teuthology.orchestra.run.smithi107.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T06:34:02.649 INFO:teuthology.orchestra.run.smithi107.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T06:34:02.650 INFO:teuthology.orchestra.run.smithi107.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T06:34:02.664 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T06:34:02.664 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T06:34:02.684 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T06:34:02.712 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T06:34:02.718 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T06:34:02.725 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T06:34:02.726 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T06:34:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T06:34:02.795 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T06:34:02.796 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T06:34:02.796 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T06:34:02.797 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T06:34:02.799 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T06:34:02.807 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T06:34:02.807 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T06:34:02.808 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T06:34:02.888 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T06:34:02.888 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T06:34:08.089 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 16.0 MB in 2s (7,496 kB/s) 2024-09-16T06:34:08.229 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 16.0 MB in 2s (7,541 kB/s) 2024-09-16T06:34:08.253 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 16.0 MB in 2s (7,371 kB/s) 2024-09-16T06:34:08.286 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 16.0 MB in 2s (7,622 kB/s) 2024-09-16T06:34:09.492 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-16T06:34:09.509 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:09.566 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-16T06:34:09.573 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-16T06:34:09.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:09.689 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-16T06:34:09.732 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-16T06:34:09.745 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-09-16T06:34:09.750 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:09.762 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:09.783 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2024-09-16T06:34:09.784 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2024-09-16T06:34:09.814 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-16T06:34:09.867 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-09-16T06:34:09.896 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-09-16T06:34:09.897 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-09-16T06:34:09.974 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:09.974 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:09.974 INFO:teuthology.orchestra.run.smithi107.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:09.975 INFO:teuthology.orchestra.run.smithi107.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:09.975 INFO:teuthology.orchestra.run.smithi107.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T06:34:09.975 INFO:teuthology.orchestra.run.smithi107.stdout: python2.7-minimal 2024-09-16T06:34:09.975 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:09.976 INFO:teuthology.orchestra.run.smithi107.stdout:The following additional packages will be installed: 2024-09-16T06:34:09.976 INFO:teuthology.orchestra.run.smithi107.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:09.976 INFO:teuthology.orchestra.run.smithi107.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T06:34:09.976 INFO:teuthology.orchestra.run.smithi107.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:09.978 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2024-09-16T06:34:09.978 INFO:teuthology.orchestra.run.smithi107.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T06:34:09.978 INFO:teuthology.orchestra.run.smithi107.stdout:Recommended packages: 2024-09-16T06:34:09.978 INFO:teuthology.orchestra.run.smithi107.stdout: thermald 2024-09-16T06:34:10.025 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-09-16T06:34:10.026 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-09-16T06:34:10.030 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2024-09-16T06:34:10.030 INFO:teuthology.orchestra.run.smithi107.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.030 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T06:34:10.030 INFO:teuthology.orchestra.run.smithi107.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.031 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages will be upgraded: 2024-09-16T06:34:10.032 INFO:teuthology.orchestra.run.smithi107.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T06:34:10.062 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-09-16T06:34:10.063 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-09-16T06:34:10.081 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:10.081 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:10.081 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:10.082 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:10.082 INFO:teuthology.orchestra.run.smithi100.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T06:34:10.082 INFO:teuthology.orchestra.run.smithi100.stdout: python2.7-minimal 2024-09-16T06:34:10.082 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:10.084 INFO:teuthology.orchestra.run.smithi107.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:10.084 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 113 MB of archives. 2024-09-16T06:34:10.084 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T06:34:10.084 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T06:34:10.085 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-09-16T06:34:10.086 INFO:teuthology.orchestra.run.smithi100.stdout: thermald 2024-09-16T06:34:10.143 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-09-16T06:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T06:34:10.144 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.145 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-09-16T06:34:10.146 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T06:34:10.213 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:10.213 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:10.213 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:10.213 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:10.213 INFO:teuthology.orchestra.run.smithi115.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T06:34:10.214 INFO:teuthology.orchestra.run.smithi115.stdout: python2.7-minimal 2024-09-16T06:34:10.214 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:10.215 INFO:teuthology.orchestra.run.smithi115.stdout:The following additional packages will be installed: 2024-09-16T06:34:10.215 INFO:teuthology.orchestra.run.smithi115.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.215 INFO:teuthology.orchestra.run.smithi115.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T06:34:10.215 INFO:teuthology.orchestra.run.smithi115.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.217 INFO:teuthology.orchestra.run.smithi115.stdout:Suggested packages: 2024-09-16T06:34:10.217 INFO:teuthology.orchestra.run.smithi115.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T06:34:10.217 INFO:teuthology.orchestra.run.smithi115.stdout:Recommended packages: 2024-09-16T06:34:10.217 INFO:teuthology.orchestra.run.smithi115.stdout: thermald 2024-09-16T06:34:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:10.232 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:10.232 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:10.233 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:10.233 INFO:teuthology.orchestra.run.smithi117.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T06:34:10.233 INFO:teuthology.orchestra.run.smithi117.stdout: python2.7-minimal 2024-09-16T06:34:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-09-16T06:34:10.234 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.234 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T06:34:10.235 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-09-16T06:34:10.236 INFO:teuthology.orchestra.run.smithi117.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T06:34:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-09-16T06:34:10.236 INFO:teuthology.orchestra.run.smithi117.stdout: thermald 2024-09-16T06:34:10.270 INFO:teuthology.orchestra.run.smithi115.stdout:The following NEW packages will be installed: 2024-09-16T06:34:10.270 INFO:teuthology.orchestra.run.smithi115.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.270 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T06:34:10.271 INFO:teuthology.orchestra.run.smithi115.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.272 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages will be upgraded: 2024-09-16T06:34:10.272 INFO:teuthology.orchestra.run.smithi115.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T06:34:10.278 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-09-16T06:34:10.279 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T06:34:10.279 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T06:34:10.279 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T06:34:10.280 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2024-09-16T06:34:10.281 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T06:34:10.324 INFO:teuthology.orchestra.run.smithi115.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:10.324 INFO:teuthology.orchestra.run.smithi115.stdout:Need to get 113 MB of archives. 2024-09-16T06:34:10.324 INFO:teuthology.orchestra.run.smithi115.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T06:34:10.324 INFO:teuthology.orchestra.run.smithi115.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T06:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 113 MB of archives. 2024-09-16T06:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T06:34:10.325 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T06:34:10.332 INFO:teuthology.orchestra.run.smithi117.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:10.333 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 113 MB of archives. 2024-09-16T06:34:10.333 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T06:34:10.333 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T06:34:10.519 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T06:34:10.614 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T06:34:10.752 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T06:34:10.761 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T06:34:10.840 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T06:34:10.864 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T06:34:11.508 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T06:34:11.558 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T06:34:11.564 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T06:34:11.654 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T06:34:11.655 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T06:34:11.655 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T06:34:11.682 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T06:34:11.682 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T06:34:11.682 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T06:34:11.716 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T06:34:11.739 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T06:34:11.778 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T06:34:11.799 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T06:34:11.802 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T06:34:11.824 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T06:34:11.942 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T06:34:12.140 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 113 MB in 2s (75.0 MB/s) 2024-09-16T06:34:12.163 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 113 MB in 2s (66.6 MB/s) 2024-09-16T06:34:12.205 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 113 MB in 2s (73.4 MB/s) 2024-09-16T06:34:12.292 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T06:34:12.309 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T06:34:12.341 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T06:34:12.390 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T06:34:14.282 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T06:34:14.286 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:14.321 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T06:34:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:14.396 INFO:teuthology.orchestra.run.smithi115.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T06:34:14.398 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:14.401 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:14.420 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:14.471 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:14.523 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T06:34:14.523 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T06:34:14.524 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T06:34:14.863 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T06:34:14.942 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T06:34:15.321 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 113 MB in 5s (23.7 MB/s) 2024-09-16T06:34:15.502 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T06:34:16.422 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T06:34:16.443 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T06:34:17.004 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:17.041 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:17.122 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T06:34:17.140 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T06:34:17.143 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:17.162 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:17.232 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:17.236 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:17.565 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T06:34:17.569 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:17.649 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:17.709 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T06:34:17.710 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T06:34:17.729 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:17.730 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:17.766 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:17.768 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:19.975 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T06:34:19.996 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:20.079 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:20.536 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T06:34:20.557 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:20.594 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:23.633 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:23.751 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:24.010 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:24.059 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:24.135 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:24.196 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:24.383 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:24.475 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:24.499 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T06:34:24.510 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T06:34:24.536 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:24.549 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:24.588 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:24.886 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:24.900 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T06:34:24.923 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T06:34:24.966 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:24.997 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:25.300 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T06:34:25.323 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T06:34:25.367 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:26.995 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:27.103 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:27.420 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:27.549 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:27.860 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T06:34:27.882 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T06:34:27.926 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T06:34:30.335 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:30.373 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:30.619 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T06:34:30.655 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:30.693 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:31.523 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T06:34:31.561 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:31.597 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:32.951 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:33.074 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:33.418 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:33.456 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:33.562 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:33.578 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:33.688 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T06:34:33.815 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:33.869 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T06:34:33.895 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:33.905 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T06:34:33.943 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:34.016 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:34.142 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T06:34:34.245 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:34.400 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:34.516 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:34.899 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:35.037 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:35.163 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T06:34:35.266 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:35.301 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:35.565 INFO:teuthology.orchestra.run.smithi115.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:35.565 INFO:teuthology.orchestra.run.smithi115.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:35.565 INFO:teuthology.orchestra.run.smithi115.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:35.565 INFO:teuthology.orchestra.run.smithi115.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:35.731 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:36.579 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T06:34:36.598 INFO:teuthology.orchestra.run.smithi117.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:36.599 INFO:teuthology.orchestra.run.smithi117.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:36.599 INFO:teuthology.orchestra.run.smithi117.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:36.599 INFO:teuthology.orchestra.run.smithi117.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:36.710 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:36.713 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T06:34:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:37.438 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T06:34:37.944 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:37.957 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:38.070 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T06:34:38.197 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:38.237 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T06:34:38.363 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T06:34:38.403 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:38.489 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:38.556 INFO:teuthology.orchestra.run.smithi107.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T06:34:38.556 INFO:teuthology.orchestra.run.smithi107.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:39.342 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T06:34:39.451 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T06:34:39.521 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:39.521 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:39.521 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:39.522 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:39.552 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T06:34:39.603 INFO:teuthology.orchestra.run.smithi115.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:39.667 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:39.813 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T06:34:39.963 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T06:34:40.065 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:40.118 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T06:34:40.118 INFO:teuthology.orchestra.run.smithi117.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:41.068 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:42.469 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T06:34:42.595 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T06:34:42.722 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T06:34:42.787 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T06:34:42.788 INFO:teuthology.orchestra.run.smithi100.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:53.405 INFO:teuthology.orchestra.run.smithi107.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T06:34:53.405 INFO:teuthology.orchestra.run.smithi107.stdout:Sourcing file `/etc/default/grub' 2024-09-16T06:34:53.422 INFO:teuthology.orchestra.run.smithi107.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:53.438 INFO:teuthology.orchestra.run.smithi107.stdout:Generating grub configuration file ... 2024-09-16T06:34:53.725 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:53.745 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:54.047 INFO:teuthology.orchestra.run.smithi115.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T06:34:54.047 INFO:teuthology.orchestra.run.smithi115.stdout:Sourcing file `/etc/default/grub' 2024-09-16T06:34:54.063 INFO:teuthology.orchestra.run.smithi115.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:54.075 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:54.079 INFO:teuthology.orchestra.run.smithi115.stdout:Generating grub configuration file ... 2024-09-16T06:34:54.080 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:54.153 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:34:54.158 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:34:54.307 INFO:teuthology.orchestra.run.smithi107.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:34:54.307 INFO:teuthology.orchestra.run.smithi107.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:34:54.308 INFO:teuthology.orchestra.run.smithi107.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:34:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-09-16T06:34:54.349 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:54.369 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:54.392 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T06:34:54.392 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub' 2024-09-16T06:34:54.406 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:54.430 INFO:teuthology.orchestra.run.smithi117.stdout:Generating grub configuration file ... 2024-09-16T06:34:54.663 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:54.668 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:54.738 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:34:54.744 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:34:54.763 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:54.777 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:54.848 DEBUG:teuthology.orchestra.run.smithi107:> dpkg -s linux-image-generic 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Package: linux-image-generic 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Status: install ok installed 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Priority: optional 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Section: kernel 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Installed-Size: 21 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: amd64 2024-09-16T06:34:54.867 INFO:teuthology.orchestra.run.smithi107.stdout:Source: linux-meta 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout:Recommends: thermald 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:54.868 INFO:teuthology.orchestra.run.smithi107.stdout: available. 2024-09-16T06:34:54.868 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T06:34:54.868 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T06:34:54.868 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T06:34:54.869 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T06:34:54.869 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T06:34:54.869 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get clean 2024-09-16T06:34:54.907 INFO:teuthology.orchestra.run.smithi115.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:34:54.907 INFO:teuthology.orchestra.run.smithi115.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:34:54.907 INFO:teuthology.orchestra.run.smithi115.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:34:54.952 INFO:teuthology.orchestra.run.smithi115.stdout:done 2024-09-16T06:34:55.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2024-09-16T06:34:55.125 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:55.132 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:55.136 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:55.147 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T06:34:55.165 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:34:55.200 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:34:55.204 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:34:55.277 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T06:34:55.386 INFO:teuthology.orchestra.run.smithi117.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:34:55.386 INFO:teuthology.orchestra.run.smithi117.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:34:55.386 INFO:teuthology.orchestra.run.smithi117.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:34:55.432 INFO:teuthology.orchestra.run.smithi117.stdout:done 2024-09-16T06:34:55.471 DEBUG:teuthology.orchestra.run.smithi115:> dpkg -s linux-image-generic 2024-09-16T06:34:55.486 INFO:teuthology.orchestra.run.smithi115.stdout:Package: linux-image-generic 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Status: install ok installed 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Priority: optional 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Section: kernel 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Installed-Size: 21 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: amd64 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Source: linux-meta 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:55.487 INFO:teuthology.orchestra.run.smithi115.stdout:Recommends: thermald 2024-09-16T06:34:55.488 INFO:teuthology.orchestra.run.smithi115.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:55.488 INFO:teuthology.orchestra.run.smithi115.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:55.488 INFO:teuthology.orchestra.run.smithi115.stdout: available. 2024-09-16T06:34:55.488 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T06:34:55.488 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T06:34:55.488 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T06:34:55.488 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-16T06:34:55.488 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T06:34:55.488 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get clean 2024-09-16T06:34:55.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-09-16T06:34:55.763 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:55.763 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T06:34:55.773 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T06:34:55.790 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:34:55.936 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2024-09-16T06:34:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2024-09-16T06:34:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2024-09-16T06:34:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2024-09-16T06:34:55.964 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 21 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:55.965 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2024-09-16T06:34:55.966 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T06:34:55.966 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T06:34:55.966 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T06:34:55.966 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-16T06:34:55.966 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T06:34:55.966 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2024-09-16T06:34:56.132 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-09-16T06:34:56.252 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:56.260 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T06:34:56.277 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:34:56.372 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T06:34:56.567 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-16T06:34:56.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:56.649 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-16T06:34:56.858 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2024-09-16T06:34:56.860 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2024-09-16T06:34:57.054 INFO:teuthology.orchestra.run.smithi107.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T06:34:57.055 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:57.055 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:57.055 INFO:teuthology.orchestra.run.smithi107.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:57.055 INFO:teuthology.orchestra.run.smithi107.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:57.056 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:34:57.056 INFO:teuthology.orchestra.run.smithi107.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:34:57.056 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:57.099 INFO:teuthology.orchestra.run.smithi107.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:57.101 DEBUG:teuthology.orchestra.run.smithi107:> dpkg -s linux-image-generic 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Package: linux-image-generic 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Status: install ok installed 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Priority: optional 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Section: kernel 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Installed-Size: 21 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: amd64 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Source: linux-meta 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Recommends: thermald 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:57.117 INFO:teuthology.orchestra.run.smithi107.stdout: available. 2024-09-16T06:34:57.118 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-16T06:34:57.161 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.Nwl0IZuSVC 2024-09-16T06:34:57.161 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Nwl0IZuSVC 2024-09-16T06:34:57.169 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-16T06:34:57.186 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:57.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.Nwl0IZuSVC 2024-09-16T06:34:57.248 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-16T06:34:57.327 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.Nwl0IZuSVC is 10KB 2024-09-16T06:34:57.340 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.Nwl0IZuSVC 2024-09-16T06:34:57.345 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T06:34:57.366 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T06:34:57.366 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub' 2024-09-16T06:34:57.377 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:57.401 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:34:57.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T06:34:57.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T06:34:57.402 INFO:teuthology.orchestra.run.smithi100.stdout:Generating grub configuration file ... 2024-09-16T06:34:57.457 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-09-16T06:34:57.458 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-09-16T06:34:57.470 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T06:34:57.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo update-grub 2024-09-16T06:34:57.572 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-09-16T06:34:57.591 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:34:57.633 INFO:teuthology.orchestra.run.smithi115.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T06:34:57.633 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:57.633 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:34:57.634 INFO:teuthology.orchestra.run.smithi115.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:34:57.634 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:57.642 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-09-16T06:34:57.663 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:57.673 INFO:teuthology.orchestra.run.smithi115.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:57.675 DEBUG:teuthology.orchestra.run.smithi115:> dpkg -s linux-image-generic 2024-09-16T06:34:57.687 INFO:teuthology.orchestra.run.smithi115.stdout:Package: linux-image-generic 2024-09-16T06:34:57.687 INFO:teuthology.orchestra.run.smithi115.stdout:Status: install ok installed 2024-09-16T06:34:57.687 INFO:teuthology.orchestra.run.smithi115.stdout:Priority: optional 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Section: kernel 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Installed-Size: 21 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: amd64 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Source: linux-meta 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Recommends: thermald 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: available. 2024-09-16T06:34:57.689 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-16T06:34:57.689 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:57.739 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.d4uRDL4xnq 2024-09-16T06:34:57.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub/grub.cfg /tmp/tmp.d4uRDL4xnq 2024-09-16T06:34:57.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.d4uRDL4xnq 2024-09-16T06:34:57.828 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-09-16T06:34:57.828 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-09-16T06:34:57.919 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.d4uRDL4xnq is 10KB 2024-09-16T06:34:57.931 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.d4uRDL4xnq 2024-09-16T06:34:57.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T06:34:57.964 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:57.971 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:57.991 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:34:57.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T06:34:57.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T06:34:58.001 INFO:teuthology.orchestra.run.smithi117.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T06:34:58.001 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:34:58.001 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:34:58.001 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:34:58.001 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:34:58.002 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:34:58.002 INFO:teuthology.orchestra.run.smithi117.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:34:58.002 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:34:58.037 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:34:58.042 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:34:58.043 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2024-09-16T06:34:58.043 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:34:58.059 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T06:34:58.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo update-grub 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 21 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2024-09-16T06:34:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:58.100 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:58.100 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2024-09-16T06:34:58.100 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-09-16T06:34:58.144 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.x4a16dh19Q 2024-09-16T06:34:58.144 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub/grub.cfg /tmp/tmp.x4a16dh19Q 2024-09-16T06:34:58.199 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.x4a16dh19Q 2024-09-16T06:34:58.215 INFO:teuthology.orchestra.run.smithi100.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:34:58.215 INFO:teuthology.orchestra.run.smithi100.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:34:58.215 INFO:teuthology.orchestra.run.smithi100.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:34:58.261 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-09-16T06:34:58.329 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.x4a16dh19Q is 10KB 2024-09-16T06:34:58.341 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.x4a16dh19Q 2024-09-16T06:34:58.346 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T06:34:58.400 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:34:58.400 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T06:34:58.400 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T06:34:58.467 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T06:34:58.467 DEBUG:teuthology.orchestra.run.smithi117:> sudo update-grub 2024-09-16T06:34:58.597 INFO:teuthology.orchestra.run.smithi107.stderr:Sourcing file `/etc/default/grub' 2024-09-16T06:34:58.598 INFO:teuthology.orchestra.run.smithi107.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:58.600 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-16T06:34:58.702 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-09-16T06:34:58.718 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-09-16T06:34:58.718 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-09-16T06:34:58.718 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 21 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.15.0.121.121 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:34:58.719 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-09-16T06:34:58.720 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-09-16T06:34:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:34:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-09-16T06:34:58.720 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T06:34:58.720 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T06:34:58.720 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T06:34:58.720 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T06:34:58.720 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T06:34:58.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-09-16T06:34:58.772 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:58.778 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:58.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-09-16T06:34:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:59.003 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:59.040 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:34:59.045 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:34:59.168 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:34:59.169 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T06:34:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:34:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:34:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:34:59.195 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-16T06:34:59.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-16T06:34:59.238 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T06:34:59.279 INFO:teuthology.orchestra.run.smithi115.stderr:Sourcing file `/etc/default/grub' 2024-09-16T06:34:59.280 INFO:teuthology.orchestra.run.smithi115.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:59.283 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-16T06:34:59.319 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:34:59.466 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:34:59.472 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:34:59.708 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:34:59.713 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:34:59.748 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:34:59.753 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:34:59.868 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub' 2024-09-16T06:34:59.870 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:34:59.876 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-09-16T06:34:59.879 INFO:teuthology.orchestra.run.smithi115.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:34:59.879 INFO:teuthology.orchestra.run.smithi115.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:34:59.879 INFO:teuthology.orchestra.run.smithi115.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:34:59.894 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-16T06:34:59.903 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-16T06:35:00.057 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:35:00.063 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:35:00.311 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:35:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:35:00.359 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:35:00.364 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:35:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:35:00.500 INFO:teuthology.orchestra.run.smithi117.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:35:00.500 INFO:teuthology.orchestra.run.smithi117.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:35:00.528 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-09-16T06:35:00.541 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-09-16T06:35:00.654 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-16T06:35:00.672 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T06:35:00.723 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-16T06:35:00.913 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-09-16T06:35:00.914 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-09-16T06:35:01.082 INFO:teuthology.orchestra.run.smithi100.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:35:01.083 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:35:01.121 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T06:35:01.123 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-09-16T06:35:01.180 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-09-16T06:35:01.180 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 21 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.15.0.121.121 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-09-16T06:35:01.181 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-09-16T06:35:01.182 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-16T06:35:01.226 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.ibfqarenva 2024-09-16T06:35:01.227 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ibfqarenva 2024-09-16T06:35:01.282 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.ibfqarenva 2024-09-16T06:35:01.405 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.ibfqarenva is 10KB 2024-09-16T06:35:01.417 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.ibfqarenva 2024-09-16T06:35:01.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T06:35:01.478 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:35:01.478 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T06:35:01.478 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T06:35:01.544 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T06:35:01.544 DEBUG:teuthology.orchestra.run.smithi100:> sudo update-grub 2024-09-16T06:35:02.848 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-09-16T06:35:02.850 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T06:35:02.852 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-16T06:35:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T06:35:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T06:35:03.269 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T06:35:03.274 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T06:35:03.315 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T06:35:03.320 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T06:35:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T06:35:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T06:35:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T06:35:03.464 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-16T06:35:03.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-16T06:35:29.225 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T06:35:29.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:35:29.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:29.906 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-16T06:35:29.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:35:29.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:30.544 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-16T06:35:30.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:35:30.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:33.479 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T06:35:33.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T06:35:33.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:47.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-16T06:35:48.413 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-16T06:35:48.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-16T06:35:52.001 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-16T06:35:56.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:35:56.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:57.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:35:57.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:57.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:35:57.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:35:59.709 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-16T06:36:00.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-16T06:36:00.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-16T06:36:01.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T06:36:01.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:04.061 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-16T06:36:11.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:36:11.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:12.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:36:12.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:12.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:36:12.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:16.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T06:36:16.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:30.145 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-16T06:36:30.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-16T06:36:31.425 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-16T06:36:45.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:36:45.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:45.161 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-16T06:36:45.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:36:45.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:46.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:36:46.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:36:46.429 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-16T06:36:46.484 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2024-09-16T06:36:47.965 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-16T06:36:48.548 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T06:36:48.548 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-16T06:36:48.549 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-16T06:36:48.595 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-121-generic 2024-09-16T06:36:48.595 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T06:36:48.595 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T06:36:48.595 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T06:36:49.596 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T06:36:49.596 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T06:36:49.700 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-09-16T06:36:49.721 DEBUG:teuthology.parallel:result is None 2024-09-16T06:36:50.571 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-16T06:36:51.134 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:36:51.134 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.15.0-121-generic"... 2024-09-16T06:36:51.134 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-16T06:36:51.180 INFO:teuthology.orchestra.run.smithi115.stdout:5.15.0-121-generic 2024-09-16T06:36:51.180 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T06:36:51.180 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T06:36:51.180 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T06:36:52.181 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-16T06:36:52.181 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T06:36:52.253 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-16T06:36:52.270 DEBUG:teuthology.parallel:result is None 2024-09-16T06:37:03.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:37:03.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:37:03.453 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-16T06:37:03.989 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:37:03.989 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-16T06:37:03.989 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-16T06:37:04.035 INFO:teuthology.orchestra.run.smithi107.stdout:5.15.0-121-generic 2024-09-16T06:37:04.036 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T06:37:04.036 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T06:37:04.036 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T06:37:04.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:37:04.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:37:04.659 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-16T06:37:05.037 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T06:37:05.037 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T06:37:05.111 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-09-16T06:37:05.129 DEBUG:teuthology.parallel:result is None 2024-09-16T06:37:05.317 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:37:05.317 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.15.0-121-generic"... 2024-09-16T06:37:05.317 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-16T06:37:05.364 INFO:teuthology.orchestra.run.smithi117.stdout:5.15.0-121-generic 2024-09-16T06:37:05.364 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T06:37:05.364 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T06:37:05.364 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T06:37:06.364 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-16T06:37:06.365 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T06:37:06.735 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-09-16T06:37:06.757 DEBUG:teuthology.parallel:result is None 2024-09-16T06:37:06.757 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T06:37:06.765 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T06:37:06.765 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T06:37:06.768 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T06:37:06.771 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T06:37:06.774 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T06:37:06.779 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T06:37:06.804 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T06:37:06.837 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T06:37:06.838 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T06:37:06.841 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T06:37:06.844 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T06:37:06.847 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T06:37:06.900 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T06:37:06.907 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T06:37:06.907 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T06:37:06.909 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T06:37:06.912 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T06:37:06.914 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T06:37:06.927 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.928 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.934 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.935 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.935 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.941 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.942 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.949 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T06:37:06.951 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T06:37:06.958 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T06:37:06.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T06:37:06.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T06:37:06.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T06:37:06.989 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T06:37:07.003 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T06:37:07.012 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T06:37:07.013 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T06:37:07.036 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T06:37:07.040 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T06:37:07.045 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T06:37:07.052 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:37:07.087 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:37:07.135 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:37:07.135 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T06:37:07.207 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:37:07.213 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:37:07.260 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:37:07.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T06:37:07.331 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:37:07.337 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:37:07.384 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:37:07.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T06:37:07.450 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T06:37:07.457 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T06:37:07.504 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:37:07.504 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T06:37:07.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-16T06:37:07.570 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-16T06:37:07.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-16T06:37:07.575 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-16T06:37:07.657 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T06:37:07.665 INFO:teuthology.task.internal:Starting timer... 2024-09-16T06:37:07.665 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T06:37:07.701 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T06:37:07.732 DEBUG:teuthology.task.selinux:Excluding smithi100: OS 'ubuntu' does not support SELinux 2024-09-16T06:37:07.733 DEBUG:teuthology.task.selinux:Excluding smithi107: OS 'ubuntu' does not support SELinux 2024-09-16T06:37:07.733 DEBUG:teuthology.task.selinux:Excluding smithi115: OS 'ubuntu' does not support SELinux 2024-09-16T06:37:07.733 DEBUG:teuthology.task.selinux:Excluding smithi117: OS 'ubuntu' does not support SELinux 2024-09-16T06:37:07.733 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T06:37:07.733 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-16T06:37:07.733 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T06:37:07.733 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T06:37:07.766 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T06:37:07.769 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T06:37:07.903 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T06:37:07.916 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T06:37:07.917 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi107.front.sepia.ceph.com,smithi115.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T06:44:44.788 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-09-16T06:44:44.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T06:44:44.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:44:44.875 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-16T06:44:44.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T06:44:44.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:44:44.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:44:45.038 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-16T06:44:45.117 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-16T06:44:45.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:44:45.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:44:45.206 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-16T06:44:45.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-16T06:44:45.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:44:45.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T06:44:45.369 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-16T06:44:45.440 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-16T06:44:45.442 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T06:44:45.453 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T06:44:45.453 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T06:44:45.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:44:45.457 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T06:44:45.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:44:45.459 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T06:44:45.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:44:45.462 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T06:44:45.462 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:44:45.488 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T06:44:45.488 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Command line: ntpd -gq 2024-09-16T06:44:45.488 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: ---------------------------------------------------- 2024-09-16T06:44:45.489 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T06:44:45.489 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T06:44:45.489 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: corporation. Support and training for ntp-4 are 2024-09-16T06:44:45.490 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: available at https://www.nwtime.org/support 2024-09-16T06:44:45.490 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: ---------------------------------------------------- 2024-09-16T06:44:45.490 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: proto: precision = 0.060 usec (-24) 2024-09-16T06:44:45.490 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: basedate set to 2022-02-04 2024-09-16T06:44:45.490 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: gps base set to 2022-02-06 (week 2196) 2024-09-16T06:44:45.491 INFO:teuthology.orchestra.run.smithi107.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.491 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.492 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.492 INFO:teuthology.orchestra.run.smithi107.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.494 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T06:44:45.494 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T06:44:45.494 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T06:44:45.494 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listen normally on 3 enp3s0f1 172.21.15.107:123 2024-09-16T06:44:45.495 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listen normally on 4 lo [::1]:123 2024-09-16T06:44:45.495 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:47eb%5]:123 2024-09-16T06:44:45.495 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:45 ntpd[18452]: Listening on routing socket on fd #22 for interface updates 2024-09-16T06:44:45.498 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T06:44:45.498 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Command line: ntpd -gq 2024-09-16T06:44:45.498 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: ---------------------------------------------------- 2024-09-16T06:44:45.498 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T06:44:45.498 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T06:44:45.498 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: corporation. Support and training for ntp-4 are 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: available at https://www.nwtime.org/support 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: ---------------------------------------------------- 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: proto: precision = 0.045 usec (-24) 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: basedate set to 2022-02-04 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: gps base set to 2022-02-06 (week 2196) 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi100.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T06:44:45.499 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Command line: ntpd -gq 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: ---------------------------------------------------- 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: corporation. Support and training for ntp-4 are 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: available at https://www.nwtime.org/support 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: ---------------------------------------------------- 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: proto: precision = 0.060 usec (-24) 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: basedate set to 2022-02-04 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: gps base set to 2022-02-06 (week 2196) 2024-09-16T06:44:45.500 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.501 INFO:teuthology.orchestra.run.smithi115.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.501 INFO:teuthology.orchestra.run.smithi115.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.501 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.501 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.502 INFO:teuthology.orchestra.run.smithi100.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listen normally on 3 enp3s0f1 172.21.15.115:123 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listen normally on 4 lo [::1]:123 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce8f%5]:123 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:45 ntpd[18498]: Listening on routing socket on fd #22 for interface updates 2024-09-16T06:44:45.504 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T06:44:45.505 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T06:44:45.505 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T06:44:45.505 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listen normally on 3 enp3s0f1 172.21.15.100:123 2024-09-16T06:44:45.505 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listen normally on 4 lo [::1]:123 2024-09-16T06:44:45.505 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d733%5]:123 2024-09-16T06:44:45.505 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:45 ntpd[18496]: Listening on routing socket on fd #22 for interface updates 2024-09-16T06:44:45.524 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Command line: ntpd -gq 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: ---------------------------------------------------- 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: corporation. Support and training for ntp-4 are 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: available at https://www.nwtime.org/support 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: ---------------------------------------------------- 2024-09-16T06:44:45.525 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: proto: precision = 0.045 usec (-24) 2024-09-16T06:44:45.526 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: basedate set to 2022-02-04 2024-09-16T06:44:45.526 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: gps base set to 2022-02-06 (week 2196) 2024-09-16T06:44:45.526 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T06:44:45.526 INFO:teuthology.orchestra.run.smithi117.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.527 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T06:44:45.527 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T06:44:45.527 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T06:44:45.527 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T06:44:45.527 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listen normally on 3 enp3s0f1 172.21.15.117:123 2024-09-16T06:44:45.528 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listen normally on 4 lo [::1]:123 2024-09-16T06:44:45.528 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4463%5]:123 2024-09-16T06:44:45.528 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:45 ntpd[18495]: Listening on routing socket on fd #22 for interface updates 2024-09-16T06:44:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:46 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:46 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:46.489 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:46 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:46.489 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:46 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:46.489 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:46 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:46.489 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:46 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:46.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:46 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:46.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:46 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:47.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:47.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:47 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:47 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:47 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:47 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:47 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:47 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:47 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:48.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:48.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:48.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:48 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:48 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:48.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:48.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:48.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:48.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:48.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:48 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:49 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:49 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:49 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:49 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:49 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:49 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:49 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.492 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:49 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:49.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:49 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:49 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:49 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:49 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:49 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:49.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:49 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:50.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:50 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:50 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:50 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:50 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:50 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:50 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:50.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:50 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:50.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:50 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:51 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:51 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:51 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:51 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:51 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:51 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:51 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.493 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:51 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:51.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:51 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:51 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:51 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:51.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:51 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:51.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:51 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:51.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:51 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:52.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:52.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:52 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:52.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.490 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:52.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:52.491 INFO:teuthology.orchestra.run.smithi115.stdout:16 Sep 06:44:52 ntpd[18498]: ntpd: time slew -0.002191 s 2024-09-16T06:44:52.491 INFO:teuthology.orchestra.run.smithi115.stdout:ntpd: time slew -0.002191s 2024-09-16T06:44:52.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T06:44:52.491 INFO:teuthology.orchestra.run.smithi115.stderr:16 Sep 06:44:52 ntpd[18498]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T06:44:52.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:52.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:52 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:52.538 INFO:teuthology.orchestra.run.smithi115.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:44:52.538 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================== 2024-09-16T06:44:52.538 INFO:teuthology.orchestra.run.smithi115.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:52.539 INFO:teuthology.orchestra.run.smithi115.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:52.539 INFO:teuthology.orchestra.run.smithi115.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:52.539 INFO:teuthology.orchestra.run.smithi115.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:53.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.488 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:53.490 INFO:teuthology.orchestra.run.smithi107.stdout:16 Sep 06:44:53 ntpd[18452]: ntpd: time slew +0.001614 s 2024-09-16T06:44:53.490 INFO:teuthology.orchestra.run.smithi107.stdout:ntpd: time slew +0.001614s 2024-09-16T06:44:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T06:44:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:16 Sep 06:44:53 ntpd[18452]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T06:44:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:53 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:53 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:53 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:53 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.491 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:53 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:53.492 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:53 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:53 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:53 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:53 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:53 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:53 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:53 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:44:53.533 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2024-09-16T06:44:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:54.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:54.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:54.488 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stdout:16 Sep 06:44:54 ntpd[18496]: ntpd: time slew -0.000391 s 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stdout:ntpd: time slew -0.000391s 2024-09-16T06:44:54.489 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T06:44:54.490 INFO:teuthology.orchestra.run.smithi100.stderr:16 Sep 06:44:54 ntpd[18496]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T06:44:54.526 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T06:44:54.527 INFO:teuthology.orchestra.run.smithi117.stdout:16 Sep 06:44:54 ntpd[18495]: ntpd: time slew -0.001453 s 2024-09-16T06:44:54.527 INFO:teuthology.orchestra.run.smithi117.stdout:ntpd: time slew -0.001453s 2024-09-16T06:44:54.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T06:44:54.527 INFO:teuthology.orchestra.run.smithi117.stderr:16 Sep 06:44:54 ntpd[18495]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T06:44:54.547 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:44:54.548 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-09-16T06:44:54.548 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.548 INFO:teuthology.orchestra.run.smithi100.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.548 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.548 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.580 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:44:54.580 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-09-16T06:44:54.580 INFO:teuthology.orchestra.run.smithi117.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.580 INFO:teuthology.orchestra.run.smithi117.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.580 INFO:teuthology.orchestra.run.smithi117.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.580 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:44:54.581 INFO:teuthology.run_tasks:Running task install... 2024-09-16T06:44:54.594 DEBUG:teuthology.task.install:project ceph 2024-09-16T06:44:54.595 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}} 2024-09-16T06:44:54.595 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T06:44:54.595 INFO:teuthology.task.install:Using flavor: default 2024-09-16T06:44:54.612 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T06:44:54.612 INFO:teuthology.task.install:extra packages: [] 2024-09-16T06:44:54.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-key list | grep Ceph 2024-09-16T06:44:54.614 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-key list | grep Ceph 2024-09-16T06:44:54.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-key list | grep Ceph 2024-09-16T06:44:54.615 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-key list | grep Ceph 2024-09-16T06:44:54.679 INFO:teuthology.orchestra.run.smithi107.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T06:44:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T06:44:54.680 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T06:44:54.681 INFO:teuthology.orchestra.run.smithi115.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T06:44:54.706 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T06:44:54.706 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T06:44:54.706 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T06:44:54.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:44:54.713 INFO:teuthology.orchestra.run.smithi115.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T06:44:54.713 INFO:teuthology.orchestra.run.smithi115.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T06:44:54.713 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T06:44:54.713 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T06:44:54.713 INFO:teuthology.orchestra.run.smithi107.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T06:44:54.714 INFO:teuthology.orchestra.run.smithi107.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T06:44:54.714 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T06:44:54.714 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:44:54.715 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T06:44:54.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:44:54.716 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T06:44:54.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:44:54.855 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default/ 2024-09-16T06:44:54.855 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:54.872 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default/ 2024-09-16T06:44:54.873 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:54.876 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default/ 2024-09-16T06:44:54.876 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:54.879 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default/ 2024-09-16T06:44:54.879 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:54.980 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:44:54.980 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T06:44:54.984 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:44:54.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T06:44:54.995 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:44:54.995 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T06:44:55.000 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:44:55.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T06:44:55.004 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-09-16T06:44:55.013 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2024-09-16T06:44:55.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-09-16T06:44:55.034 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-09-16T06:44:55.182 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:44:55.194 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T06:44:55.194 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T06:44:55.198 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:44:55.207 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:44:55.209 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T06:44:55.225 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T06:44:55.227 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:44:55.236 INFO:teuthology.orchestra.run.smithi115.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T06:44:55.236 INFO:teuthology.orchestra.run.smithi107.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T06:44:55.246 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T06:44:55.255 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T06:44:55.257 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T06:44:55.257 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T06:44:55.268 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T06:44:55.277 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T06:44:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T06:44:55.280 INFO:teuthology.orchestra.run.smithi115.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T06:44:55.289 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-09-16T06:44:55.296 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:44:55.299 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T06:44:55.301 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-09-16T06:44:55.311 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:44:55.312 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T06:44:55.312 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:44:55.312 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-09-16T06:44:55.317 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-09-16T06:44:55.320 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-09-16T06:44:55.326 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T06:44:55.329 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-09-16T06:44:55.341 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-09-16T06:44:55.347 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-09-16T06:44:55.375 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T06:44:55.383 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T06:44:55.730 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:44:55.738 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T06:44:55.771 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 158 kB in 1s (275 kB/s) 2024-09-16T06:44:55.783 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 158 kB in 1s (280 kB/s) 2024-09-16T06:44:55.900 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 157 kB in 1s (223 kB/s) 2024-09-16T06:44:55.904 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 286 kB in 1s (403 kB/s) 2024-09-16T06:44:56.801 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-16T06:44:56.816 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-09-16T06:44:56.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1jammy cephadm=18.2.4-845-ge4e227b3-1jammy ceph-mds=18.2.4-845-ge4e227b3-1jammy ceph-mgr=18.2.4-845-ge4e227b3-1jammy ceph-common=18.2.4-845-ge4e227b3-1jammy ceph-fuse=18.2.4-845-ge4e227b3-1jammy ceph-test=18.2.4-845-ge4e227b3-1jammy ceph-volume=18.2.4-845-ge4e227b3-1jammy radosgw=18.2.4-845-ge4e227b3-1jammy python3-rados=18.2.4-845-ge4e227b3-1jammy python3-rgw=18.2.4-845-ge4e227b3-1jammy python3-cephfs=18.2.4-845-ge4e227b3-1jammy python3-rbd=18.2.4-845-ge4e227b3-1jammy libcephfs2=18.2.4-845-ge4e227b3-1jammy libcephfs-dev=18.2.4-845-ge4e227b3-1jammy librados2=18.2.4-845-ge4e227b3-1jammy librbd1=18.2.4-845-ge4e227b3-1jammy rbd-fuse=18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:56.831 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1jammy cephadm=18.2.4-845-ge4e227b3-1jammy ceph-mds=18.2.4-845-ge4e227b3-1jammy ceph-mgr=18.2.4-845-ge4e227b3-1jammy ceph-common=18.2.4-845-ge4e227b3-1jammy ceph-fuse=18.2.4-845-ge4e227b3-1jammy ceph-test=18.2.4-845-ge4e227b3-1jammy ceph-volume=18.2.4-845-ge4e227b3-1jammy radosgw=18.2.4-845-ge4e227b3-1jammy python3-rados=18.2.4-845-ge4e227b3-1jammy python3-rgw=18.2.4-845-ge4e227b3-1jammy python3-cephfs=18.2.4-845-ge4e227b3-1jammy python3-rbd=18.2.4-845-ge4e227b3-1jammy libcephfs2=18.2.4-845-ge4e227b3-1jammy libcephfs-dev=18.2.4-845-ge4e227b3-1jammy librados2=18.2.4-845-ge4e227b3-1jammy librbd1=18.2.4-845-ge4e227b3-1jammy rbd-fuse=18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:56.880 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-16T06:44:56.893 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-09-16T06:44:56.921 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-16T06:44:56.939 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1jammy cephadm=18.2.4-845-ge4e227b3-1jammy ceph-mds=18.2.4-845-ge4e227b3-1jammy ceph-mgr=18.2.4-845-ge4e227b3-1jammy ceph-common=18.2.4-845-ge4e227b3-1jammy ceph-fuse=18.2.4-845-ge4e227b3-1jammy ceph-test=18.2.4-845-ge4e227b3-1jammy ceph-volume=18.2.4-845-ge4e227b3-1jammy radosgw=18.2.4-845-ge4e227b3-1jammy python3-rados=18.2.4-845-ge4e227b3-1jammy python3-rgw=18.2.4-845-ge4e227b3-1jammy python3-cephfs=18.2.4-845-ge4e227b3-1jammy python3-rbd=18.2.4-845-ge4e227b3-1jammy libcephfs2=18.2.4-845-ge4e227b3-1jammy libcephfs-dev=18.2.4-845-ge4e227b3-1jammy librados2=18.2.4-845-ge4e227b3-1jammy librbd1=18.2.4-845-ge4e227b3-1jammy rbd-fuse=18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:56.965 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-16T06:44:56.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1jammy cephadm=18.2.4-845-ge4e227b3-1jammy ceph-mds=18.2.4-845-ge4e227b3-1jammy ceph-mgr=18.2.4-845-ge4e227b3-1jammy ceph-common=18.2.4-845-ge4e227b3-1jammy ceph-fuse=18.2.4-845-ge4e227b3-1jammy ceph-test=18.2.4-845-ge4e227b3-1jammy ceph-volume=18.2.4-845-ge4e227b3-1jammy radosgw=18.2.4-845-ge4e227b3-1jammy python3-rados=18.2.4-845-ge4e227b3-1jammy python3-rgw=18.2.4-845-ge4e227b3-1jammy python3-cephfs=18.2.4-845-ge4e227b3-1jammy python3-rbd=18.2.4-845-ge4e227b3-1jammy libcephfs2=18.2.4-845-ge4e227b3-1jammy libcephfs-dev=18.2.4-845-ge4e227b3-1jammy librados2=18.2.4-845-ge4e227b3-1jammy librbd1=18.2.4-845-ge4e227b3-1jammy rbd-fuse=18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:44:57.004 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-16T06:44:57.036 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-16T06:44:57.091 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2024-09-16T06:44:57.092 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2024-09-16T06:44:57.101 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-09-16T06:44:57.102 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-09-16T06:44:57.215 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-09-16T06:44:57.215 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-09-16T06:44:57.225 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-09-16T06:44:57.226 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-09-16T06:44:57.285 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:44:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:44:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:44:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:44:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:44:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:44:57.286 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:44:57.286 INFO:teuthology.orchestra.run.smithi107.stdout:The following additional packages will be installed: 2024-09-16T06:44:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T06:44:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T06:44:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-16T06:44:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T06:44:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T06:44:57.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-16T06:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-watchdog 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout:Recommended packages: 2024-09-16T06:44:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: btrfs-tools 2024-09-16T06:44:57.298 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:44:57.299 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:44:57.299 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:44:57.299 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:44:57.299 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:44:57.299 INFO:teuthology.orchestra.run.smithi117.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:44:57.300 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:44:57.300 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-09-16T06:44:57.300 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T06:44:57.300 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T06:44:57.301 INFO:teuthology.orchestra.run.smithi117.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T06:44:57.302 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.303 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: python3-watchdog 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-09-16T06:44:57.304 INFO:teuthology.orchestra.run.smithi117.stdout: btrfs-tools 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-16T06:44:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-16T06:44:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-16T06:44:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-16T06:44:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: socat xmlstarlet 2024-09-16T06:44:57.402 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-09-16T06:44:57.402 INFO:teuthology.orchestra.run.smithi117.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T06:44:57.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T06:44:57.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T06:44:57.403 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-16T06:44:57.403 INFO:teuthology.orchestra.run.smithi117.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-16T06:44:57.403 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-16T06:44:57.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-16T06:44:57.405 INFO:teuthology.orchestra.run.smithi117.stdout: socat xmlstarlet 2024-09-16T06:44:57.409 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:44:57.409 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:44:57.409 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:44:57.409 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:44:57.410 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:44:57.410 INFO:teuthology.orchestra.run.smithi115.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:44:57.410 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:44:57.411 INFO:teuthology.orchestra.run.smithi115.stdout:The following additional packages will be installed: 2024-09-16T06:44:57.411 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T06:44:57.411 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T06:44:57.411 INFO:teuthology.orchestra.run.smithi115.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-16T06:44:57.411 INFO:teuthology.orchestra.run.smithi115.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T06:44:57.412 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-16T06:44:57.412 INFO:teuthology.orchestra.run.smithi115.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T06:44:57.412 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.414 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T06:44:57.415 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T06:44:57.416 INFO:teuthology.orchestra.run.smithi100.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-16T06:44:57.417 INFO:teuthology.orchestra.run.smithi100.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi100.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi100.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi100.stdout: python3-watchdog 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi100.stdout: btrfs-tools 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi115.stdout:Suggested packages: 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi115.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-16T06:44:57.418 INFO:teuthology.orchestra.run.smithi115.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout: python3-watchdog 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout:Recommended packages: 2024-09-16T06:44:57.419 INFO:teuthology.orchestra.run.smithi115.stdout: btrfs-tools 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-16T06:44:57.518 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-16T06:44:57.519 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi100.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi100.stdout: socat xmlstarlet 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi115.stdout:The following NEW packages will be installed: 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi115.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T06:44:57.520 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-16T06:44:57.521 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-16T06:44:57.522 INFO:teuthology.orchestra.run.smithi115.stdout: socat xmlstarlet 2024-09-16T06:44:57.536 INFO:teuthology.orchestra.run.smithi107.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T06:44:57.536 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 163 MB of archives. 2024-09-16T06:44:57.536 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-16T06:44:57.536 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1jammy [3,451 kB] 2024-09-16T06:44:57.547 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T06:44:57.548 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 163 MB of archives. 2024-09-16T06:44:57.548 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-16T06:44:57.548 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1jammy [3,451 kB] 2024-09-16T06:44:57.576 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T06:44:57.577 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 163 MB of archives. 2024-09-16T06:44:57.577 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-16T06:44:57.577 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-16T06:44:57.579 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-16T06:44:57.592 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-16T06:44:57.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T06:44:57.659 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1jammy [3,451 kB] 2024-09-16T06:44:57.666 INFO:teuthology.orchestra.run.smithi115.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T06:44:57.667 INFO:teuthology.orchestra.run.smithi115.stdout:Need to get 163 MB of archives. 2024-09-16T06:44:57.667 INFO:teuthology.orchestra.run.smithi115.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-16T06:44:57.667 INFO:teuthology.orchestra.run.smithi115.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1jammy [3,451 kB] 2024-09-16T06:44:57.671 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T06:44:57.674 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T06:44:57.689 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T06:44:57.706 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-16T06:44:57.741 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T06:44:57.745 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1jammy [3,108 kB] 2024-09-16T06:44:57.746 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T06:44:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1jammy [3,108 kB] 2024-09-16T06:44:57.757 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T06:44:57.761 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T06:44:57.762 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T06:44:57.762 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T06:44:57.771 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T06:44:57.771 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T06:44:57.772 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T06:44:57.773 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T06:44:57.779 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T06:44:57.780 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T06:44:57.796 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T06:44:57.797 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T06:44:57.797 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T06:44:57.798 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T06:44:57.800 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T06:44:57.801 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T06:44:57.801 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T06:44:57.802 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T06:44:57.813 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T06:44:57.814 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T06:44:57.830 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T06:44:57.832 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T06:44:57.832 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T06:44:57.833 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T06:44:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T06:44:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T06:44:57.839 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T06:44:57.840 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T06:44:57.847 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T06:44:57.849 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T06:44:57.850 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1jammy [683 kB] 2024-09-16T06:44:57.858 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T06:44:57.859 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T06:44:57.860 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T06:44:57.861 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T06:44:57.865 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1jammy [357 kB] 2024-09-16T06:44:57.865 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T06:44:57.866 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T06:44:57.866 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T06:44:57.867 INFO:teuthology.orchestra.run.smithi100.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T06:44:57.868 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1jammy [683 kB] 2024-09-16T06:44:57.876 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T06:44:57.881 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1jammy [357 kB] 2024-09-16T06:44:57.889 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1jammy [32.8 kB] 2024-09-16T06:44:57.890 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1jammy [180 kB] 2024-09-16T06:44:57.894 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1jammy [63.7 kB] 2024-09-16T06:44:57.896 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1jammy [333 kB] 2024-09-16T06:44:57.903 INFO:teuthology.orchestra.run.smithi107.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1jammy [5,934 kB] 2024-09-16T06:44:57.908 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1jammy [32.8 kB] 2024-09-16T06:44:57.909 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1jammy [180 kB] 2024-09-16T06:44:57.913 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1jammy [63.7 kB] 2024-09-16T06:44:57.915 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1jammy [333 kB] 2024-09-16T06:44:57.920 INFO:teuthology.orchestra.run.smithi100.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T06:44:57.921 INFO:teuthology.orchestra.run.smithi100.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T06:44:57.921 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1jammy [5,934 kB] 2024-09-16T06:44:57.922 INFO:teuthology.orchestra.run.smithi100.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T06:44:57.922 INFO:teuthology.orchestra.run.smithi100.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T06:44:57.923 INFO:teuthology.orchestra.run.smithi100.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T06:44:57.923 INFO:teuthology.orchestra.run.smithi100.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T06:44:57.923 INFO:teuthology.orchestra.run.smithi100.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T06:44:57.925 INFO:teuthology.orchestra.run.smithi100.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T06:44:57.926 INFO:teuthology.orchestra.run.smithi100.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T06:44:57.939 INFO:teuthology.orchestra.run.smithi100.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T06:44:57.956 INFO:teuthology.orchestra.run.smithi100.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T06:44:57.958 INFO:teuthology.orchestra.run.smithi100.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T06:44:57.958 INFO:teuthology.orchestra.run.smithi100.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T06:44:57.973 INFO:teuthology.orchestra.run.smithi100.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T06:44:57.973 INFO:teuthology.orchestra.run.smithi100.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T06:44:57.974 INFO:teuthology.orchestra.run.smithi100.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T06:44:57.974 INFO:teuthology.orchestra.run.smithi100.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T06:44:57.974 INFO:teuthology.orchestra.run.smithi100.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T06:44:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T06:44:57.975 INFO:teuthology.orchestra.run.smithi100.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T06:44:57.982 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T06:44:57.991 INFO:teuthology.orchestra.run.smithi100.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T06:44:57.994 INFO:teuthology.orchestra.run.smithi100.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T06:44:57.996 INFO:teuthology.orchestra.run.smithi100.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T06:44:57.997 INFO:teuthology.orchestra.run.smithi100.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T06:44:58.009 INFO:teuthology.orchestra.run.smithi100.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T06:44:58.011 INFO:teuthology.orchestra.run.smithi100.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T06:44:58.014 INFO:teuthology.orchestra.run.smithi100.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T06:44:58.018 INFO:teuthology.orchestra.run.smithi100.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T06:44:58.021 INFO:teuthology.orchestra.run.smithi100.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T06:44:58.021 INFO:teuthology.orchestra.run.smithi100.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T06:44:58.025 INFO:teuthology.orchestra.run.smithi100.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T06:44:58.029 INFO:teuthology.orchestra.run.smithi107.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T06:44:58.032 INFO:teuthology.orchestra.run.smithi100.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T06:44:58.032 INFO:teuthology.orchestra.run.smithi100.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T06:44:58.033 INFO:teuthology.orchestra.run.smithi100.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T06:44:58.044 INFO:teuthology.orchestra.run.smithi107.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T06:44:58.048 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T06:44:58.066 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T06:44:58.116 INFO:teuthology.orchestra.run.smithi107.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T06:44:58.130 INFO:teuthology.orchestra.run.smithi107.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1jammy [112 kB] 2024-09-16T06:44:58.132 INFO:teuthology.orchestra.run.smithi107.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1jammy [445 kB] 2024-09-16T06:44:58.135 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T06:44:58.141 INFO:teuthology.orchestra.run.smithi107.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1jammy [22.0 MB] 2024-09-16T06:44:58.152 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T06:44:58.155 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1jammy [112 kB] 2024-09-16T06:44:58.158 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1jammy [445 kB] 2024-09-16T06:44:58.168 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T06:44:58.168 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1jammy [22.0 MB] 2024-09-16T06:44:58.238 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T06:44:58.311 INFO:teuthology.orchestra.run.smithi107.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T06:44:58.320 INFO:teuthology.orchestra.run.smithi107.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T06:44:58.329 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T06:44:58.339 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T06:44:58.347 INFO:teuthology.orchestra.run.smithi107.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T06:44:58.356 INFO:teuthology.orchestra.run.smithi107.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T06:44:58.357 INFO:teuthology.orchestra.run.smithi107.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T06:44:58.359 INFO:teuthology.orchestra.run.smithi107.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T06:44:58.367 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T06:44:58.376 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T06:44:58.376 INFO:teuthology.orchestra.run.smithi107.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T06:44:58.377 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T06:44:58.378 INFO:teuthology.orchestra.run.smithi107.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T06:44:58.378 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T06:44:58.379 INFO:teuthology.orchestra.run.smithi107.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T06:44:58.394 INFO:teuthology.orchestra.run.smithi107.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T06:44:58.396 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T06:44:58.397 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T06:44:58.398 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T06:44:58.411 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T06:44:58.432 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T06:44:58.442 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T06:44:58.463 INFO:teuthology.orchestra.run.smithi107.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T06:44:58.463 INFO:teuthology.orchestra.run.smithi107.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T06:44:58.464 INFO:teuthology.orchestra.run.smithi107.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T06:44:58.466 INFO:teuthology.orchestra.run.smithi107.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T06:44:58.466 INFO:teuthology.orchestra.run.smithi107.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T06:44:58.467 INFO:teuthology.orchestra.run.smithi107.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T06:44:58.473 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T06:44:58.477 INFO:teuthology.orchestra.run.smithi107.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T06:44:58.478 INFO:teuthology.orchestra.run.smithi107.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T06:44:58.479 INFO:teuthology.orchestra.run.smithi107.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T06:44:58.482 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T06:44:58.482 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T06:44:58.483 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T06:44:58.483 INFO:teuthology.orchestra.run.smithi100.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1jammy [3,108 kB] 2024-09-16T06:44:58.484 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T06:44:58.484 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T06:44:58.485 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T06:44:58.490 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T06:44:58.491 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T06:44:58.493 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T06:44:58.510 INFO:teuthology.orchestra.run.smithi115.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T06:44:58.511 INFO:teuthology.orchestra.run.smithi115.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T06:44:58.512 INFO:teuthology.orchestra.run.smithi115.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T06:44:58.527 INFO:teuthology.orchestra.run.smithi115.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T06:44:58.528 INFO:teuthology.orchestra.run.smithi115.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T06:44:58.529 INFO:teuthology.orchestra.run.smithi115.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T06:44:58.531 INFO:teuthology.orchestra.run.smithi115.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T06:44:58.557 INFO:teuthology.orchestra.run.smithi107.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T06:44:58.559 INFO:teuthology.orchestra.run.smithi107.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T06:44:58.560 INFO:teuthology.orchestra.run.smithi107.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T06:44:58.561 INFO:teuthology.orchestra.run.smithi107.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T06:44:58.563 INFO:teuthology.orchestra.run.smithi107.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T06:44:58.565 INFO:teuthology.orchestra.run.smithi107.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T06:44:58.574 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T06:44:58.576 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T06:44:58.577 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T06:44:58.578 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T06:44:58.580 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T06:44:58.581 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T06:44:58.582 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T06:44:58.593 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T06:44:58.596 INFO:teuthology.orchestra.run.smithi115.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T06:44:58.596 INFO:teuthology.orchestra.run.smithi115.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T06:44:58.597 INFO:teuthology.orchestra.run.smithi115.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T06:44:58.639 INFO:teuthology.orchestra.run.smithi107.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T06:44:58.650 INFO:teuthology.orchestra.run.smithi107.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T06:44:58.650 INFO:teuthology.orchestra.run.smithi107.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T06:44:58.652 INFO:teuthology.orchestra.run.smithi107.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T06:44:58.656 INFO:teuthology.orchestra.run.smithi117.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T06:44:58.657 INFO:teuthology.orchestra.run.smithi117.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T06:44:58.675 INFO:teuthology.orchestra.run.smithi117.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T06:44:58.677 INFO:teuthology.orchestra.run.smithi115.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T06:44:58.677 INFO:teuthology.orchestra.run.smithi117.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T06:44:58.678 INFO:teuthology.orchestra.run.smithi115.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T06:44:58.678 INFO:teuthology.orchestra.run.smithi115.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T06:44:58.681 INFO:teuthology.orchestra.run.smithi117.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T06:44:58.683 INFO:teuthology.orchestra.run.smithi117.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T06:44:58.684 INFO:teuthology.orchestra.run.smithi115.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T06:44:58.685 INFO:teuthology.orchestra.run.smithi115.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T06:44:58.686 INFO:teuthology.orchestra.run.smithi115.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T06:44:58.687 INFO:teuthology.orchestra.run.smithi115.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T06:44:58.720 INFO:teuthology.orchestra.run.smithi107.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T06:44:58.722 INFO:teuthology.orchestra.run.smithi107.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T06:44:58.726 INFO:teuthology.orchestra.run.smithi107.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T06:44:58.728 INFO:teuthology.orchestra.run.smithi107.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T06:44:58.731 INFO:teuthology.orchestra.run.smithi107.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T06:44:58.733 INFO:teuthology.orchestra.run.smithi107.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T06:44:58.738 INFO:teuthology.orchestra.run.smithi117.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T06:44:58.739 INFO:teuthology.orchestra.run.smithi117.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T06:44:58.744 INFO:teuthology.orchestra.run.smithi117.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T06:44:58.745 INFO:teuthology.orchestra.run.smithi117.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T06:44:58.758 INFO:teuthology.orchestra.run.smithi115.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T06:44:58.759 INFO:teuthology.orchestra.run.smithi115.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T06:44:58.760 INFO:teuthology.orchestra.run.smithi115.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T06:44:58.801 INFO:teuthology.orchestra.run.smithi107.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T06:44:58.802 INFO:teuthology.orchestra.run.smithi107.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T06:44:58.804 INFO:teuthology.orchestra.run.smithi107.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T06:44:58.809 INFO:teuthology.orchestra.run.smithi107.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T06:44:58.819 INFO:teuthology.orchestra.run.smithi117.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T06:44:58.823 INFO:teuthology.orchestra.run.smithi117.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T06:44:58.840 INFO:teuthology.orchestra.run.smithi115.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T06:44:58.841 INFO:teuthology.orchestra.run.smithi115.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T06:44:58.843 INFO:teuthology.orchestra.run.smithi115.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T06:44:58.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T06:44:58.854 INFO:teuthology.orchestra.run.smithi115.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T06:44:58.855 INFO:teuthology.orchestra.run.smithi115.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T06:44:58.856 INFO:teuthology.orchestra.run.smithi115.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T06:44:58.914 INFO:teuthology.orchestra.run.smithi115.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1jammy [3,108 kB] 2024-09-16T06:44:58.921 INFO:teuthology.orchestra.run.smithi115.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T06:44:58.925 INFO:teuthology.orchestra.run.smithi115.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T06:44:58.927 INFO:teuthology.orchestra.run.smithi115.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T06:44:58.994 INFO:teuthology.orchestra.run.smithi117.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T06:44:58.996 INFO:teuthology.orchestra.run.smithi117.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T06:44:58.999 INFO:teuthology.orchestra.run.smithi117.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T06:44:58.999 INFO:teuthology.orchestra.run.smithi117.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T06:44:59.001 INFO:teuthology.orchestra.run.smithi117.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T06:44:59.002 INFO:teuthology.orchestra.run.smithi117.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T06:44:59.003 INFO:teuthology.orchestra.run.smithi117.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T06:44:59.003 INFO:teuthology.orchestra.run.smithi115.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T06:44:59.004 INFO:teuthology.orchestra.run.smithi115.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T06:44:59.007 INFO:teuthology.orchestra.run.smithi117.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T06:44:59.007 INFO:teuthology.orchestra.run.smithi117.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T06:44:59.009 INFO:teuthology.orchestra.run.smithi115.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T06:44:59.010 INFO:teuthology.orchestra.run.smithi115.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T06:44:59.012 INFO:teuthology.orchestra.run.smithi115.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T06:44:59.016 INFO:teuthology.orchestra.run.smithi115.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T06:44:59.018 INFO:teuthology.orchestra.run.smithi107.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T06:44:59.020 INFO:teuthology.orchestra.run.smithi100.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1jammy [683 kB] 2024-09-16T06:44:59.079 INFO:teuthology.orchestra.run.smithi117.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T06:44:59.082 INFO:teuthology.orchestra.run.smithi100.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1jammy [357 kB] 2024-09-16T06:44:59.082 INFO:teuthology.orchestra.run.smithi107.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1jammy [5,005 kB] 2024-09-16T06:44:59.082 INFO:teuthology.orchestra.run.smithi107.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T06:44:59.085 INFO:teuthology.orchestra.run.smithi107.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T06:44:59.086 INFO:teuthology.orchestra.run.smithi107.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T06:44:59.088 INFO:teuthology.orchestra.run.smithi107.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T06:44:59.089 INFO:teuthology.orchestra.run.smithi107.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T06:44:59.090 INFO:teuthology.orchestra.run.smithi107.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T06:44:59.096 INFO:teuthology.orchestra.run.smithi107.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T06:44:59.096 INFO:teuthology.orchestra.run.smithi107.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T06:44:59.116 INFO:teuthology.orchestra.run.smithi100.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1jammy [32.8 kB] 2024-09-16T06:44:59.117 INFO:teuthology.orchestra.run.smithi100.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1jammy [180 kB] 2024-09-16T06:44:59.150 INFO:teuthology.orchestra.run.smithi117.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1jammy [5,005 kB] 2024-09-16T06:44:59.150 INFO:teuthology.orchestra.run.smithi117.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T06:44:59.153 INFO:teuthology.orchestra.run.smithi117.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T06:44:59.154 INFO:teuthology.orchestra.run.smithi100.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1jammy [63.7 kB] 2024-09-16T06:44:59.154 INFO:teuthology.orchestra.run.smithi117.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T06:44:59.160 INFO:teuthology.orchestra.run.smithi100.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1jammy [333 kB] 2024-09-16T06:44:59.165 INFO:teuthology.orchestra.run.smithi107.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T06:44:59.171 INFO:teuthology.orchestra.run.smithi115.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T06:44:59.172 INFO:teuthology.orchestra.run.smithi115.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T06:44:59.174 INFO:teuthology.orchestra.run.smithi115.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T06:44:59.175 INFO:teuthology.orchestra.run.smithi115.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T06:44:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T06:44:59.176 INFO:teuthology.orchestra.run.smithi115.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T06:44:59.177 INFO:teuthology.orchestra.run.smithi115.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T06:44:59.180 INFO:teuthology.orchestra.run.smithi107.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1jammy [241 kB] 2024-09-16T06:44:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T06:44:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T06:44:59.187 INFO:teuthology.orchestra.run.smithi117.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T06:44:59.187 INFO:teuthology.orchestra.run.smithi117.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T06:44:59.188 INFO:teuthology.orchestra.run.smithi117.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T06:44:59.189 INFO:teuthology.orchestra.run.smithi117.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T06:44:59.190 INFO:teuthology.orchestra.run.smithi117.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T06:44:59.191 INFO:teuthology.orchestra.run.smithi117.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T06:44:59.206 INFO:teuthology.orchestra.run.smithi100.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1jammy [5,934 kB] 2024-09-16T06:44:59.208 INFO:teuthology.orchestra.run.smithi107.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1jammy [130 kB] 2024-09-16T06:44:59.211 INFO:teuthology.orchestra.run.smithi107.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1jammy [1,494 kB] 2024-09-16T06:44:59.213 INFO:teuthology.orchestra.run.smithi107.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T06:44:59.217 INFO:teuthology.orchestra.run.smithi107.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T06:44:59.218 INFO:teuthology.orchestra.run.smithi107.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T06:44:59.242 INFO:teuthology.orchestra.run.smithi117.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T06:44:59.243 INFO:teuthology.orchestra.run.smithi117.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T06:44:59.249 INFO:teuthology.orchestra.run.smithi117.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T06:44:59.251 INFO:teuthology.orchestra.run.smithi115.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1jammy [683 kB] 2024-09-16T06:44:59.255 INFO:teuthology.orchestra.run.smithi115.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T06:44:59.265 INFO:teuthology.orchestra.run.smithi107.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1jammy [6,085 kB] 2024-09-16T06:44:59.267 INFO:teuthology.orchestra.run.smithi107.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T06:44:59.268 INFO:teuthology.orchestra.run.smithi107.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T06:44:59.269 INFO:teuthology.orchestra.run.smithi107.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T06:44:59.269 INFO:teuthology.orchestra.run.smithi117.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1jammy [241 kB] 2024-09-16T06:44:59.270 INFO:teuthology.orchestra.run.smithi107.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T06:44:59.271 INFO:teuthology.orchestra.run.smithi107.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T06:44:59.272 INFO:teuthology.orchestra.run.smithi107.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T06:44:59.275 INFO:teuthology.orchestra.run.smithi117.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1jammy [130 kB] 2024-09-16T06:44:59.277 INFO:teuthology.orchestra.run.smithi117.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1jammy [1,494 kB] 2024-09-16T06:44:59.293 INFO:teuthology.orchestra.run.smithi115.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T06:44:59.296 INFO:teuthology.orchestra.run.smithi115.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T06:44:59.297 INFO:teuthology.orchestra.run.smithi115.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T06:44:59.323 INFO:teuthology.orchestra.run.smithi117.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T06:44:59.326 INFO:teuthology.orchestra.run.smithi117.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T06:44:59.328 INFO:teuthology.orchestra.run.smithi117.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T06:44:59.329 INFO:teuthology.orchestra.run.smithi107.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T06:44:59.330 INFO:teuthology.orchestra.run.smithi107.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T06:44:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1jammy [6,085 kB] 2024-09-16T06:44:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T06:44:59.337 INFO:teuthology.orchestra.run.smithi107.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T06:44:59.340 INFO:teuthology.orchestra.run.smithi115.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1jammy [357 kB] 2024-09-16T06:44:59.341 INFO:teuthology.orchestra.run.smithi117.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T06:44:59.351 INFO:teuthology.orchestra.run.smithi117.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T06:44:59.356 INFO:teuthology.orchestra.run.smithi115.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T06:44:59.356 INFO:teuthology.orchestra.run.smithi115.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T06:44:59.357 INFO:teuthology.orchestra.run.smithi115.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T06:44:59.358 INFO:teuthology.orchestra.run.smithi115.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T06:44:59.358 INFO:teuthology.orchestra.run.smithi117.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T06:44:59.358 INFO:teuthology.orchestra.run.smithi115.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T06:44:59.359 INFO:teuthology.orchestra.run.smithi115.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T06:44:59.385 INFO:teuthology.orchestra.run.smithi115.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1jammy [32.8 kB] 2024-09-16T06:44:59.386 INFO:teuthology.orchestra.run.smithi115.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1jammy [180 kB] 2024-09-16T06:44:59.390 INFO:teuthology.orchestra.run.smithi115.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1jammy [63.7 kB] 2024-09-16T06:44:59.392 INFO:teuthology.orchestra.run.smithi115.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1jammy [333 kB] 2024-09-16T06:44:59.405 INFO:teuthology.orchestra.run.smithi117.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T06:44:59.408 INFO:teuthology.orchestra.run.smithi117.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T06:44:59.410 INFO:teuthology.orchestra.run.smithi107.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T06:44:59.413 INFO:teuthology.orchestra.run.smithi107.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T06:44:59.415 INFO:teuthology.orchestra.run.smithi107.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T06:44:59.417 INFO:teuthology.orchestra.run.smithi115.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T06:44:59.418 INFO:teuthology.orchestra.run.smithi115.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T06:44:59.423 INFO:teuthology.orchestra.run.smithi107.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T06:44:59.424 INFO:teuthology.orchestra.run.smithi117.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T06:44:59.425 INFO:teuthology.orchestra.run.smithi115.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T06:44:59.429 INFO:teuthology.orchestra.run.smithi107.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T06:44:59.436 INFO:teuthology.orchestra.run.smithi115.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1jammy [5,934 kB] 2024-09-16T06:44:59.439 INFO:teuthology.orchestra.run.smithi107.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T06:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T06:44:59.486 INFO:teuthology.orchestra.run.smithi117.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T06:44:59.488 INFO:teuthology.orchestra.run.smithi117.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T06:44:59.491 INFO:teuthology.orchestra.run.smithi107.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T06:44:59.497 INFO:teuthology.orchestra.run.smithi107.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T06:44:59.499 INFO:teuthology.orchestra.run.smithi115.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T06:44:59.502 INFO:teuthology.orchestra.run.smithi115.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T06:44:59.504 INFO:teuthology.orchestra.run.smithi115.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T06:44:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T06:44:59.512 INFO:teuthology.orchestra.run.smithi115.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T06:44:59.517 INFO:teuthology.orchestra.run.smithi115.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T06:44:59.527 INFO:teuthology.orchestra.run.smithi115.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T06:44:59.534 INFO:teuthology.orchestra.run.smithi115.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T06:44:59.544 INFO:teuthology.orchestra.run.smithi107.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1jammy [22.4 MB] 2024-09-16T06:44:59.573 INFO:teuthology.orchestra.run.smithi107.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T06:44:59.575 INFO:teuthology.orchestra.run.smithi107.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T06:44:59.580 INFO:teuthology.orchestra.run.smithi115.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T06:44:59.584 INFO:teuthology.orchestra.run.smithi115.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T06:44:59.600 INFO:teuthology.orchestra.run.smithi115.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T06:44:59.618 INFO:teuthology.orchestra.run.smithi117.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1jammy [22.4 MB] 2024-09-16T06:44:59.662 INFO:teuthology.orchestra.run.smithi115.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T06:44:59.664 INFO:teuthology.orchestra.run.smithi115.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T06:44:59.964 INFO:teuthology.orchestra.run.smithi115.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1jammy [112 kB] 2024-09-16T06:44:59.967 INFO:teuthology.orchestra.run.smithi115.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1jammy [445 kB] 2024-09-16T06:45:00.017 INFO:teuthology.orchestra.run.smithi115.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1jammy [22.0 MB] 2024-09-16T06:45:00.214 INFO:teuthology.orchestra.run.smithi100.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1jammy [112 kB] 2024-09-16T06:45:00.258 INFO:teuthology.orchestra.run.smithi100.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1jammy [445 kB] 2024-09-16T06:45:00.328 INFO:teuthology.orchestra.run.smithi100.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1jammy [22.0 MB] 2024-09-16T06:45:00.696 INFO:teuthology.orchestra.run.smithi107.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1jammy [14.1 kB] 2024-09-16T06:45:00.696 INFO:teuthology.orchestra.run.smithi107.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1jammy [834 kB] 2024-09-16T06:45:00.712 INFO:teuthology.orchestra.run.smithi107.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1jammy [2,159 kB] 2024-09-16T06:45:00.773 INFO:teuthology.orchestra.run.smithi107.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1jammy [240 kB] 2024-09-16T06:45:00.786 INFO:teuthology.orchestra.run.smithi107.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1jammy [145 kB] 2024-09-16T06:45:00.789 INFO:teuthology.orchestra.run.smithi107.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1jammy [2,233 kB] 2024-09-16T06:45:00.812 INFO:teuthology.orchestra.run.smithi117.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1jammy [14.1 kB] 2024-09-16T06:45:00.812 INFO:teuthology.orchestra.run.smithi117.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1jammy [834 kB] 2024-09-16T06:45:00.829 INFO:teuthology.orchestra.run.smithi117.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1jammy [2,159 kB] 2024-09-16T06:45:00.871 INFO:teuthology.orchestra.run.smithi107.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1jammy [8,625 kB] 2024-09-16T06:45:00.882 INFO:teuthology.orchestra.run.smithi117.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1jammy [240 kB] 2024-09-16T06:45:00.886 INFO:teuthology.orchestra.run.smithi117.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1jammy [145 kB] 2024-09-16T06:45:00.889 INFO:teuthology.orchestra.run.smithi117.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1jammy [2,233 kB] 2024-09-16T06:45:01.119 INFO:teuthology.orchestra.run.smithi117.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1jammy [8,625 kB] 2024-09-16T06:45:01.340 INFO:teuthology.orchestra.run.smithi115.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1jammy [5,005 kB] 2024-09-16T06:45:01.346 INFO:teuthology.orchestra.run.smithi107.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1jammy [14.3 kB] 2024-09-16T06:45:01.346 INFO:teuthology.orchestra.run.smithi107.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1jammy [42.8 MB] 2024-09-16T06:45:01.505 INFO:teuthology.orchestra.run.smithi115.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1jammy [241 kB] 2024-09-16T06:45:01.510 INFO:teuthology.orchestra.run.smithi115.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1jammy [130 kB] 2024-09-16T06:45:01.513 INFO:teuthology.orchestra.run.smithi115.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1jammy [1,494 kB] 2024-09-16T06:45:01.532 INFO:teuthology.orchestra.run.smithi117.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1jammy [14.3 kB] 2024-09-16T06:45:01.532 INFO:teuthology.orchestra.run.smithi117.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1jammy [42.8 MB] 2024-09-16T06:45:01.576 INFO:teuthology.orchestra.run.smithi115.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1jammy [6,085 kB] 2024-09-16T06:45:01.874 INFO:teuthology.orchestra.run.smithi115.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1jammy [22.4 MB] 2024-09-16T06:45:02.986 INFO:teuthology.orchestra.run.smithi115.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1jammy [14.1 kB] 2024-09-16T06:45:02.987 INFO:teuthology.orchestra.run.smithi115.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1jammy [834 kB] 2024-09-16T06:45:03.003 INFO:teuthology.orchestra.run.smithi115.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1jammy [2,159 kB] 2024-09-16T06:45:03.070 INFO:teuthology.orchestra.run.smithi115.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1jammy [240 kB] 2024-09-16T06:45:03.097 INFO:teuthology.orchestra.run.smithi115.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1jammy [145 kB] 2024-09-16T06:45:03.100 INFO:teuthology.orchestra.run.smithi115.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1jammy [2,233 kB] 2024-09-16T06:45:03.182 INFO:teuthology.orchestra.run.smithi115.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1jammy [8,625 kB] 2024-09-16T06:45:03.585 INFO:teuthology.orchestra.run.smithi115.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1jammy [14.3 kB] 2024-09-16T06:45:03.585 INFO:teuthology.orchestra.run.smithi115.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1jammy [42.8 MB] 2024-09-16T06:45:03.952 INFO:teuthology.orchestra.run.smithi107.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1jammy [122 kB] 2024-09-16T06:45:03.955 INFO:teuthology.orchestra.run.smithi107.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1jammy [38.3 kB] 2024-09-16T06:45:03.956 INFO:teuthology.orchestra.run.smithi107.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1jammy [11.7 MB] 2024-09-16T06:45:04.126 INFO:teuthology.orchestra.run.smithi117.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1jammy [122 kB] 2024-09-16T06:45:04.198 INFO:teuthology.orchestra.run.smithi117.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1jammy [38.3 kB] 2024-09-16T06:45:04.199 INFO:teuthology.orchestra.run.smithi117.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1jammy [11.7 MB] 2024-09-16T06:45:04.475 INFO:teuthology.orchestra.run.smithi107.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1jammy [93.1 kB] 2024-09-16T06:45:04.641 INFO:teuthology.orchestra.run.smithi117.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1jammy [93.1 kB] 2024-09-16T06:45:04.952 INFO:teuthology.orchestra.run.smithi100.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1jammy [5,005 kB] 2024-09-16T06:45:05.048 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 163 MB in 7s (23.0 MB/s) 2024-09-16T06:45:05.162 INFO:teuthology.orchestra.run.smithi115.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1jammy [122 kB] 2024-09-16T06:45:05.165 INFO:teuthology.orchestra.run.smithi115.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1jammy [38.3 kB] 2024-09-16T06:45:05.165 INFO:teuthology.orchestra.run.smithi115.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1jammy [11.7 MB] 2024-09-16T06:45:05.180 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-16T06:45:05.225 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 163 MB in 7s (22.5 MB/s) 2024-09-16T06:45:05.272 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T06:45:05.278 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-16T06:45:05.316 INFO:teuthology.orchestra.run.smithi100.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1jammy [241 kB] 2024-09-16T06:45:05.339 INFO:teuthology.orchestra.run.smithi115.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1jammy [93.1 kB] 2024-09-16T06:45:05.339 INFO:teuthology.orchestra.run.smithi100.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1jammy [130 kB] 2024-09-16T06:45:05.343 INFO:teuthology.orchestra.run.smithi100.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1jammy [1,494 kB] 2024-09-16T06:45:05.356 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-16T06:45:05.370 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:45:05.440 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T06:45:05.445 INFO:teuthology.orchestra.run.smithi100.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1jammy [6,085 kB] 2024-09-16T06:45:05.446 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-16T06:45:05.539 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:45:05.707 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T06:45:05.733 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T06:45:05.773 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:45:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1jammy [22.4 MB] 2024-09-16T06:45:05.883 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T06:45:05.906 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 163 MB in 8s (20.8 MB/s) 2024-09-16T06:45:05.909 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T06:45:05.949 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:45:06.003 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-16T06:45:06.085 INFO:teuthology.orchestra.run.smithi115.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T06:45:06.090 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-16T06:45:06.110 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T06:45:06.137 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T06:45:06.177 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:06.194 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:45:06.244 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T06:45:06.270 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T06:45:06.303 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:06.505 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T06:45:06.531 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T06:45:06.555 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T06:45:06.571 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:45:06.581 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T06:45:06.622 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:45:06.647 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T06:45:06.674 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T06:45:06.714 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:45:06.992 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T06:45:07.009 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T06:45:07.018 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:07.035 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T06:45:07.075 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:07.083 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:07.159 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T06:45:07.185 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:07.234 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:07.370 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T06:45:07.392 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T06:45:07.437 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:45:07.479 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T06:45:07.505 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:07.541 INFO:teuthology.orchestra.run.smithi100.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1jammy [14.1 kB] 2024-09-16T06:45:07.542 INFO:teuthology.orchestra.run.smithi100.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1jammy [834 kB] 2024-09-16T06:45:07.558 INFO:teuthology.orchestra.run.smithi100.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1jammy [2,159 kB] 2024-09-16T06:45:07.596 INFO:teuthology.orchestra.run.smithi100.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1jammy [240 kB] 2024-09-16T06:45:07.607 INFO:teuthology.orchestra.run.smithi100.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1jammy [145 kB] 2024-09-16T06:45:07.623 INFO:teuthology.orchestra.run.smithi100.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1jammy [2,233 kB] 2024-09-16T06:45:07.734 INFO:teuthology.orchestra.run.smithi100.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1jammy [8,625 kB] 2024-09-16T06:45:07.787 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:07.865 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T06:45:07.879 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T06:45:07.892 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:07.905 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:07.940 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:07.945 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:08.107 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T06:45:08.133 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:08.165 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:08.193 INFO:teuthology.orchestra.run.smithi100.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1jammy [14.3 kB] 2024-09-16T06:45:08.193 INFO:teuthology.orchestra.run.smithi100.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1jammy [42.8 MB] 2024-09-16T06:45:08.274 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T06:45:08.300 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:08.340 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:08.352 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T06:45:08.367 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:08.410 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:08.502 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T06:45:08.528 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T06:45:08.568 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:45:08.685 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T06:45:08.702 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T06:45:08.743 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:45:08.746 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T06:45:08.773 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:08.813 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:08.897 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librados2. 2024-09-16T06:45:08.923 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:08.963 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:09.029 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librados2. 2024-09-16T06:45:09.056 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:09.087 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:09.133 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T06:45:09.159 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T06:45:09.191 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:45:09.408 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librbd1. 2024-09-16T06:45:09.434 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../009-librbd1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:09.474 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:09.511 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librados2. 2024-09-16T06:45:09.523 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librbd1. 2024-09-16T06:45:09.537 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:09.550 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../009-librbd1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:09.577 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:09.894 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T06:45:09.921 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.009 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T06:45:10.014 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librbd1. 2024-09-16T06:45:10.036 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.040 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../009-librbd1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.068 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.072 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.230 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rados. 2024-09-16T06:45:10.257 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../011-python3-rados_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.296 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.320 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rados. 2024-09-16T06:45:10.346 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../011-python3-rados_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.387 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.492 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T06:45:10.518 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.545 INFO:teuthology.orchestra.run.smithi100.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1jammy [122 kB] 2024-09-16T06:45:10.547 INFO:teuthology.orchestra.run.smithi100.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1jammy [38.3 kB] 2024-09-16T06:45:10.549 INFO:teuthology.orchestra.run.smithi100.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1jammy [11.7 MB] 2024-09-16T06:45:10.558 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.616 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T06:45:10.642 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:10.682 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.723 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T06:45:10.749 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:10.789 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.819 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rados. 2024-09-16T06:45:10.834 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../011-python3-rados_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.877 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:10.927 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T06:45:10.953 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:10.985 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1jammy [93.1 kB] 2024-09-16T06:45:11.042 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T06:45:11.068 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:11.108 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.171 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T06:45:11.187 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:11.222 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.288 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T06:45:11.315 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:11.355 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.424 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T06:45:11.439 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:11.444 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T06:45:11.471 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:11.476 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.511 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.580 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 163 MB in 13s (12.1 MB/s) 2024-09-16T06:45:11.691 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T06:45:11.718 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-16T06:45:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T06:45:11.758 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:45:11.787 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T06:45:11.806 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T06:45:11.813 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:11.816 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T06:45:11.822 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-16T06:45:11.833 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T06:45:11.854 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:11.864 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:45:11.908 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:45:12.086 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T06:45:12.112 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T06:45:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T06:45:12.300 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:45:12.372 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T06:45:12.386 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:12.519 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T06:45:12.546 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:12.556 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T06:45:12.567 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T06:45:12.571 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T06:45:12.586 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:12.594 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T06:45:12.606 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:12.634 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T06:45:12.775 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T06:45:12.790 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T06:45:12.833 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:45:12.881 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T06:45:12.895 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T06:45:12.907 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T06:45:12.910 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:12.934 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T06:45:12.945 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:12.947 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:45:12.960 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T06:45:13.009 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:45:13.178 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librgw2. 2024-09-16T06:45:13.202 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:13.266 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librgw2. 2024-09-16T06:45:13.282 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:13.325 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:13.454 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T06:45:13.481 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:13.529 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:13.531 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T06:45:13.557 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T06:45:13.597 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:45:13.739 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T06:45:13.754 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:13.789 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:13.827 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T06:45:13.842 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:13.885 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:13.915 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T06:45:13.925 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librgw2. 2024-09-16T06:45:13.941 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:13.951 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:13.973 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:13.991 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:14.067 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T06:45:14.093 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T06:45:14.133 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:45:14.222 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T06:45:14.249 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T06:45:14.288 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:45:14.310 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T06:45:14.336 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T06:45:14.376 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:45:14.445 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T06:45:14.471 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:14.502 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T06:45:14.511 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:14.517 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:14.549 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T06:45:14.560 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:14.576 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:14.616 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:14.713 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T06:45:14.740 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T06:45:14.779 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:45:14.896 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T06:45:14.923 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T06:45:14.931 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-common. 2024-09-16T06:45:14.958 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:14.963 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:45:14.998 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:15.032 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librados2. 2024-09-16T06:45:15.036 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-common. 2024-09-16T06:45:15.059 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:15.063 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:15.090 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:15.104 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:15.224 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T06:45:15.239 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:15.274 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:15.501 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librbd1. 2024-09-16T06:45:15.528 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../009-librbd1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:15.568 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:15.653 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-common. 2024-09-16T06:45:15.679 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:15.719 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:16.004 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T06:45:16.031 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:16.071 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:16.296 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-base. 2024-09-16T06:45:16.323 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:16.372 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:16.373 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rados. 2024-09-16T06:45:16.391 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-base. 2024-09-16T06:45:16.400 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../011-python3-rados_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:16.418 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:16.448 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:16.476 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:16.734 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T06:45:16.760 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:16.792 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:16.949 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-base. 2024-09-16T06:45:16.950 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T06:45:16.975 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:16.977 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T06:45:17.012 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T06:45:17.016 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:45:17.025 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:17.027 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T06:45:17.062 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:45:17.070 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T06:45:17.096 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:17.128 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:17.320 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T06:45:17.340 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T06:45:17.346 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T06:45:17.366 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T06:45:17.386 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:45:17.406 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:45:17.439 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T06:45:17.466 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:17.506 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:17.587 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T06:45:17.613 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T06:45:17.653 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:45:17.690 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T06:45:17.716 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T06:45:17.734 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T06:45:17.756 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:45:17.761 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T06:45:17.801 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:45:17.809 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T06:45:17.835 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T06:45:17.875 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:45:17.956 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T06:45:17.982 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T06:45:18.009 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T06:45:18.022 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:45:18.034 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T06:45:18.075 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:45:18.195 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T06:45:18.221 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T06:45:18.254 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T06:45:18.261 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T06:45:18.280 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T06:45:18.317 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T06:45:18.320 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:45:18.343 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T06:45:18.379 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T06:45:18.391 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:45:18.405 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T06:45:18.445 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:45:18.505 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T06:45:18.521 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:18.555 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:18.623 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T06:45:18.649 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T06:45:18.653 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T06:45:18.679 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T06:45:18.689 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:45:18.719 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:45:18.723 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T06:45:18.749 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T06:45:18.790 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T06:45:18.900 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T06:45:18.926 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T06:45:18.966 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:45:18.975 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T06:45:19.002 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T06:45:19.022 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T06:45:19.049 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T06:45:19.051 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-portend. 2024-09-16T06:45:19.067 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-16T06:45:19.067 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T06:45:19.089 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:45:19.109 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T06:45:19.269 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librgw2. 2024-09-16T06:45:19.296 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:19.336 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:19.345 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-portend. 2024-09-16T06:45:19.372 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-16T06:45:19.392 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T06:45:19.403 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T06:45:19.417 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T06:45:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T06:45:19.456 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T06:45:19.458 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T06:45:19.496 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T06:45:19.681 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T06:45:19.707 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T06:45:19.719 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-portend. 2024-09-16T06:45:19.745 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-16T06:45:19.773 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T06:45:19.786 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T06:45:19.799 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T06:45:19.826 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T06:45:19.838 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T06:45:19.853 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:19.865 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:45:19.896 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:20.072 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T06:45:20.076 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T06:45:20.098 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T06:45:20.102 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T06:45:20.138 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T06:45:20.142 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:45:20.211 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T06:45:20.216 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T06:45:20.237 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T06:45:20.242 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T06:45:20.274 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:45:20.277 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T06:45:20.425 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T06:45:20.451 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T06:45:20.454 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T06:45:20.480 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T06:45:20.491 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:45:20.510 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T06:45:20.520 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T06:45:20.536 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:20.577 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:20.589 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T06:45:20.616 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T06:45:20.655 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T06:45:20.828 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T06:45:20.840 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T06:45:20.843 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T06:45:20.866 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T06:45:20.877 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T06:45:20.907 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T06:45:20.958 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T06:45:20.985 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T06:45:20.997 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-common. 2024-09-16T06:45:21.023 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:21.025 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:45:21.063 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:21.164 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T06:45:21.190 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T06:45:21.209 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T06:45:21.230 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T06:45:21.235 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T06:45:21.276 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:45:21.328 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-mako. 2024-09-16T06:45:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T06:45:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:45:21.533 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T06:45:21.559 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T06:45:21.579 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-mako. 2024-09-16T06:45:21.600 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:45:21.605 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T06:45:21.646 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:45:21.740 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T06:45:21.767 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T06:45:21.806 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:45:21.911 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-mako. 2024-09-16T06:45:21.937 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T06:45:21.948 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T06:45:21.975 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T06:45:21.977 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:45:22.015 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:45:22.068 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T06:45:22.094 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T06:45:22.126 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:45:22.242 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T06:45:22.258 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T06:45:22.289 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T06:45:22.292 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:45:22.304 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T06:45:22.318 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-base. 2024-09-16T06:45:22.338 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:45:22.345 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:22.394 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:22.429 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webob. 2024-09-16T06:45:22.456 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T06:45:22.495 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:45:22.595 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webob. 2024-09-16T06:45:22.608 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T06:45:22.621 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T06:45:22.634 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T06:45:22.662 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:45:22.676 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:45:22.849 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T06:45:22.875 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T06:45:22.919 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:45:22.948 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T06:45:22.974 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T06:45:22.986 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-webob. 2024-09-16T06:45:23.012 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T06:45:23.014 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:45:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T06:45:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T06:45:23.052 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:45:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:45:23.293 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T06:45:23.318 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T06:45:23.350 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:45:23.392 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T06:45:23.396 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T06:45:23.419 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T06:45:23.423 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T06:45:23.459 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:45:23.469 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:45:23.477 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T06:45:23.503 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T06:45:23.543 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:45:23.670 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T06:45:23.697 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T06:45:23.737 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:45:23.737 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-paste. 2024-09-16T06:45:23.752 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T06:45:23.787 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:45:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-paste. 2024-09-16T06:45:23.828 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T06:45:23.863 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:45:23.949 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T06:45:23.976 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T06:45:24.015 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T06:45:24.015 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:45:24.030 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T06:45:24.064 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:45:24.106 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T06:45:24.122 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T06:45:24.165 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:45:24.166 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T06:45:24.181 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T06:45:24.224 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:45:24.309 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T06:45:24.335 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T06:45:24.335 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-paste. 2024-09-16T06:45:24.362 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T06:45:24.367 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:45:24.402 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:45:24.434 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T06:45:24.449 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T06:45:24.484 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:45:24.511 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T06:45:24.537 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T06:45:24.577 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:45:24.670 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T06:45:24.696 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T06:45:24.721 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T06:45:24.736 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T06:45:24.748 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T06:45:24.754 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T06:45:24.780 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T06:45:24.788 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:45:24.820 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T06:45:24.880 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T06:45:24.907 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T06:45:24.947 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T06:45:25.048 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-portend. 2024-09-16T06:45:25.074 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T06:45:25.074 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-16T06:45:25.089 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T06:45:25.114 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T06:45:25.123 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:45:25.131 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T06:45:25.158 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T06:45:25.198 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:45:25.258 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T06:45:25.285 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T06:45:25.325 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:45:25.394 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T06:45:25.409 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T06:45:25.420 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T06:45:25.435 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T06:45:25.460 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T06:45:25.467 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T06:45:25.595 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T06:45:25.621 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T06:45:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T06:45:25.688 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T06:45:25.715 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T06:45:25.736 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T06:45:25.755 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T06:45:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T06:45:25.771 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T06:45:25.795 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:45:25.798 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T06:45:25.838 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:45:25.997 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-idna. 2024-09-16T06:45:26.024 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T06:45:26.032 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-idna. 2024-09-16T06:45:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T06:45:26.064 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:45:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:45:26.174 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T06:45:26.184 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T06:45:26.200 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T06:45:26.211 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T06:45:26.240 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T06:45:26.250 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T06:45:26.342 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T06:45:26.357 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T06:45:26.392 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:45:26.430 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T06:45:26.456 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T06:45:26.504 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:45:26.510 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T06:45:26.536 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T06:45:26.568 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T06:45:26.595 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-idna. 2024-09-16T06:45:26.622 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T06:45:26.661 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests. 2024-09-16T06:45:26.662 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:45:26.677 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T06:45:26.743 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:45:26.833 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-requests. 2024-09-16T06:45:26.859 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T06:45:26.862 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T06:45:26.888 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T06:45:26.899 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:45:26.929 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:45:26.981 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T06:45:27.008 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T06:45:27.047 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:45:27.048 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T06:45:27.074 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T06:45:27.114 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:45:27.218 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T06:45:27.234 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T06:45:27.240 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-mako. 2024-09-16T06:45:27.266 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T06:45:27.276 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:45:27.307 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:45:27.367 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T06:45:27.375 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-requests. 2024-09-16T06:45:27.394 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:27.402 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T06:45:27.425 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:27.442 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:45:27.563 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T06:45:27.589 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:27.627 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T06:45:27.629 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:27.653 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T06:45:27.693 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:45:27.720 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T06:45:27.747 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T06:45:27.778 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:45:27.814 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T06:45:27.840 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:27.880 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:27.954 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T06:45:27.980 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T06:45:28.009 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T06:45:28.012 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:45:28.036 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:28.075 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T06:45:28.096 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:28.139 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:28.149 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T06:45:28.165 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T06:45:28.200 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:45:28.315 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webob. 2024-09-16T06:45:28.341 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T06:45:28.381 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:45:28.386 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T06:45:28.413 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T06:45:28.453 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:45:28.652 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T06:45:28.678 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T06:45:28.714 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T06:45:28.718 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T06:45:28.726 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T06:45:28.730 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T06:45:28.753 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T06:45:28.764 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T06:45:28.796 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:45:29.080 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T06:45:29.083 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T06:45:29.104 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T06:45:29.106 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:29.110 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:29.130 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T06:45:29.147 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:29.149 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:29.170 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:45:29.328 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T06:45:29.355 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:29.394 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:29.490 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-paste. 2024-09-16T06:45:29.516 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T06:45:29.517 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T06:45:29.543 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:29.557 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:45:29.583 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:29.705 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T06:45:29.732 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T06:45:29.772 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:45:29.893 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T06:45:29.908 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T06:45:29.951 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:45:30.033 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T06:45:30.048 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T06:45:30.082 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T06:45:30.136 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T06:45:30.163 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:30.203 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:30.254 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T06:45:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T06:45:30.320 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:45:30.345 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T06:45:30.371 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:30.402 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T06:45:30.411 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:30.429 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:30.469 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:30.855 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T06:45:30.882 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:30.921 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:30.965 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T06:45:30.992 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:31.023 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:31.558 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T06:45:31.570 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T06:45:31.585 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:31.597 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T06:45:31.625 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:31.683 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T06:45:31.915 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph. 2024-09-16T06:45:31.938 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../060-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:31.981 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:31.985 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T06:45:32.012 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T06:45:32.052 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:45:32.076 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph. 2024-09-16T06:45:32.092 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../060-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:32.134 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:32.284 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T06:45:32.310 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:32.350 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:32.415 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T06:45:32.442 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T06:45:32.479 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T06:45:32.481 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T06:45:32.506 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:32.545 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:32.628 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph. 2024-09-16T06:45:32.654 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../060-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:32.695 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:32.695 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T06:45:32.722 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:32.762 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:32.809 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-idna. 2024-09-16T06:45:32.836 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T06:45:32.876 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:45:32.924 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T06:45:32.950 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:32.990 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:33.014 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T06:45:33.040 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:33.080 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:33.132 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephadm. 2024-09-16T06:45:33.158 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../063-cephadm_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:33.162 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T06:45:33.188 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T06:45:33.198 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:33.228 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:45:33.442 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T06:45:33.468 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:33.506 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package cephadm. 2024-09-16T06:45:33.508 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:33.520 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T06:45:33.533 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../063-cephadm_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:33.542 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests. 2024-09-16T06:45:33.544 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T06:45:33.557 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T06:45:33.584 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T06:45:33.584 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:33.600 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:45:33.887 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T06:45:33.895 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T06:45:33.895 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T06:45:33.914 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T06:45:33.921 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T06:45:33.922 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:33.953 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T06:45:33.961 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:45:33.962 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:34.045 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package cephadm. 2024-09-16T06:45:34.071 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../063-cephadm_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:34.103 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:34.231 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T06:45:34.240 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T06:45:34.246 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:34.266 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:34.289 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:34.298 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:34.397 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T06:45:34.424 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T06:45:34.464 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T06:45:34.653 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T06:45:34.679 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:34.719 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:34.750 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T06:45:34.765 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:34.800 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:34.967 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-babel. 2024-09-16T06:45:34.988 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T06:45:34.994 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:35.015 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T06:45:35.033 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:35.046 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:45:35.333 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T06:45:35.336 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-babel. 2024-09-16T06:45:35.355 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T06:45:35.359 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T06:45:35.363 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:35.382 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T06:45:35.402 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:35.408 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T06:45:35.422 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:45:35.714 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T06:45:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T06:45:35.744 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T06:45:35.766 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T06:45:35.771 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:35.780 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-babel. 2024-09-16T06:45:35.780 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:45:35.793 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:35.807 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:35.811 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:35.833 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:35.846 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:36.124 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T06:45:36.140 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T06:45:36.142 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T06:45:36.168 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:36.174 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:45:36.178 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T06:45:36.197 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T06:45:36.205 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T06:45:36.209 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:36.224 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:36.244 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:45:36.264 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:36.511 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T06:45:36.514 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-py. 2024-09-16T06:45:36.528 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T06:45:36.529 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-16T06:45:36.538 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:36.555 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T06:45:36.564 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T06:45:36.569 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:36.603 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:45:36.851 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T06:45:36.876 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T06:45:36.877 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:36.902 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T06:45:36.914 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T06:45:36.914 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-py. 2024-09-16T06:45:36.917 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:36.941 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T06:45:36.941 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-16T06:45:36.942 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:36.980 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:45:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T06:45:37.241 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-py. 2024-09-16T06:45:37.245 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-toml. 2024-09-16T06:45:37.259 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T06:45:37.268 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-16T06:45:37.271 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-16T06:45:37.286 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T06:45:37.308 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T06:45:37.311 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T06:45:37.317 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:37.611 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T06:45:37.614 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T06:45:37.628 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-toml. 2024-09-16T06:45:37.638 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T06:45:37.641 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T06:45:37.655 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-16T06:45:37.678 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:37.681 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:45:37.695 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T06:45:37.946 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph. 2024-09-16T06:45:37.961 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../060-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:37.980 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-toml. 2024-09-16T06:45:37.981 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T06:45:38.004 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:38.008 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-16T06:45:38.008 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T06:45:38.009 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T06:45:38.036 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T06:45:38.047 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T06:45:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:45:38.076 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T06:45:38.332 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T06:45:38.350 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T06:45:38.358 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:38.377 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T06:45:38.384 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T06:45:38.398 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:38.404 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-routes. 2024-09-16T06:45:38.411 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T06:45:38.416 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:45:38.430 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T06:45:38.450 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T06:45:38.470 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:45:38.769 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T06:45:38.787 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T06:45:38.795 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:38.814 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T06:45:38.835 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:38.853 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T06:45:38.925 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-routes. 2024-09-16T06:45:38.951 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T06:45:38.965 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T06:45:38.987 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:45:38.992 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:39.032 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:39.156 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-routes. 2024-09-16T06:45:39.183 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T06:45:39.213 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephadm. 2024-09-16T06:45:39.222 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:45:39.239 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../063-cephadm_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:39.279 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:39.282 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T06:45:39.309 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:39.348 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:39.576 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T06:45:39.591 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T06:45:39.603 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:39.618 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T06:45:39.651 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:39.657 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T06:45:39.969 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T06:45:39.984 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:40.027 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:40.253 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T06:45:40.269 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T06:45:40.311 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:45:40.528 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T06:45:40.555 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T06:45:40.603 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:45:40.832 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T06:45:40.855 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T06:45:40.858 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T06:45:40.871 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T06:45:40.898 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:45:40.905 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:45:41.024 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-babel. 2024-09-16T06:45:41.051 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T06:45:41.090 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:45:41.098 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T06:45:41.125 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T06:45:41.165 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:45:41.210 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T06:45:41.237 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T06:45:41.268 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:45:41.427 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T06:45:41.454 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T06:45:41.475 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T06:45:41.503 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T06:45:41.519 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:45:41.543 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T06:45:41.559 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:45:41.570 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T06:45:41.571 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T06:45:41.598 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T06:45:41.609 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:45:41.637 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:45:41.896 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T06:45:41.922 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T06:45:41.954 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:45:41.979 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T06:45:42.006 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T06:45:42.022 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T06:45:42.048 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:42.054 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:45:42.088 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:42.340 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T06:45:42.367 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T06:45:42.379 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T06:45:42.406 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:45:42.407 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:42.408 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T06:45:42.435 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T06:45:42.445 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:42.475 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:45:42.645 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T06:45:42.673 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:42.703 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:42.719 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-py. 2024-09-16T06:45:42.735 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-16T06:45:42.769 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T06:45:43.031 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T06:45:43.058 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T06:45:43.081 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T06:45:43.089 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:43.109 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:43.148 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:43.376 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-toml. 2024-09-16T06:45:43.382 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T06:45:43.402 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-16T06:45:43.409 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T06:45:43.442 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T06:45:43.448 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T06:45:43.623 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T06:45:43.650 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T06:45:43.689 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T06:45:43.745 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T06:45:43.772 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T06:45:43.812 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:45:43.835 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T06:45:43.863 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T06:45:43.902 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:45:44.077 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T06:45:44.104 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T06:45:44.109 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T06:45:44.115 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T06:45:44.136 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T06:45:44.142 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T06:45:44.143 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:45:44.184 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T06:45:44.190 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T06:45:44.247 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T06:45:44.274 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T06:45:44.314 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:45:44.443 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-routes. 2024-09-16T06:45:44.459 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T06:45:44.472 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T06:45:44.493 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:45:44.499 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T06:45:44.538 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:45:44.558 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T06:45:44.586 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-16T06:45:44.625 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T06:45:44.763 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T06:45:44.790 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T06:45:44.829 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:45:44.830 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T06:45:44.833 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T06:45:44.857 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:44.859 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-16T06:45:44.896 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:44.899 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T06:45:44.945 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T06:45:44.972 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T06:45:45.011 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T06:45:45.133 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T06:45:45.160 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T06:45:45.177 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T06:45:45.193 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T06:45:45.199 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:45:45.227 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T06:45:45.331 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T06:45:45.358 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T06:45:45.398 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:45:45.511 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T06:45:45.514 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T06:45:45.538 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-16T06:45:45.541 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T06:45:45.577 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T06:45:45.580 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:45:45.692 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T06:45:45.719 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T06:45:45.759 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T06:45:45.891 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T06:45:45.914 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T06:45:45.918 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T06:45:45.941 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T06:45:45.958 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T06:45:45.980 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T06:45:46.087 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T06:45:46.114 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T06:45:46.153 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:45:46.194 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T06:45:46.221 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T06:45:46.260 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:45:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T06:45:46.309 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T06:45:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T06:45:46.336 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T06:45:46.352 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:45:46.375 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:45:46.678 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T06:45:46.705 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T06:45:46.744 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T06:45:46.772 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T06:45:46.784 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T06:45:46.799 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T06:45:46.800 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:46.833 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:46.838 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:45:46.999 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T06:45:47.015 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:47.039 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T06:45:47.057 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:47.066 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T06:45:47.097 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:45:47.136 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T06:45:47.164 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T06:45:47.201 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T06:45:47.202 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:45:47.227 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T06:45:47.267 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:45:47.385 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T06:45:47.412 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T06:45:47.451 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:45:47.564 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T06:45:47.570 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T06:45:47.591 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:47.597 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T06:45:47.630 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:45:47.652 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:45:47.778 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T06:45:47.794 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:47.796 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T06:45:47.824 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:47.836 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:47.863 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:45:47.866 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package jq. 2024-09-16T06:45:47.882 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:47.916 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T06:45:48.073 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package jq. 2024-09-16T06:45:48.089 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:48.123 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T06:45:48.144 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package socat. 2024-09-16T06:45:48.164 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T06:45:48.171 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T06:45:48.189 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T06:45:48.210 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:45:48.230 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:45:48.256 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T06:45:48.270 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:48.312 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:48.359 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package socat. 2024-09-16T06:45:48.375 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T06:45:48.417 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:45:48.505 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T06:45:48.521 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T06:45:48.563 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:45:48.575 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T06:45:48.602 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:48.641 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:45:48.729 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T06:45:48.756 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T06:45:48.795 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:45:48.869 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package jq. 2024-09-16T06:45:48.885 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:48.893 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-test. 2024-09-16T06:45:48.920 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../096-ceph-test_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:48.927 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T06:45:48.959 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-test. 2024-09-16T06:45:49.180 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package socat. 2024-09-16T06:45:49.185 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../096-ceph-test_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:49.207 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T06:45:49.246 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:45:49.252 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:49.282 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T06:45:49.308 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T06:45:49.348 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T06:45:49.753 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T06:45:49.769 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T06:45:49.811 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:45:49.816 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T06:45:49.843 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T06:45:49.882 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:45:50.139 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T06:45:50.155 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T06:45:50.197 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:45:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-test. 2024-09-16T06:45:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../096-ceph-test_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:50.518 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T06:45:50.545 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-16T06:45:50.584 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T06:45:50.913 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T06:45:50.940 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T06:45:50.979 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T06:45:51.096 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T06:45:51.123 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:51.163 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:51.291 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T06:45:51.318 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T06:45:51.345 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T06:45:51.365 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:45:51.372 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:51.411 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:51.458 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T06:45:51.474 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:51.516 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:51.644 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T06:45:51.671 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T06:45:51.707 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T06:45:51.710 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T06:45:51.734 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:51.773 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:51.894 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T06:45:51.921 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T06:45:51.961 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:52.022 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T06:45:52.049 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T06:45:52.076 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T06:45:52.088 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:45:52.091 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T06:45:52.126 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:52.176 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T06:45:52.203 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:52.242 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:52.299 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T06:45:52.315 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T06:45:52.348 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:45:52.430 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T06:45:52.446 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T06:45:52.480 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:45:52.546 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T06:45:52.574 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:52.613 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:52.686 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T06:45:52.702 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:52.735 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:52.768 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T06:45:52.796 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T06:45:52.835 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T06:45:52.875 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T06:45:52.902 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T06:45:52.941 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T06:45:53.016 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T06:45:53.044 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T06:45:53.055 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T06:45:53.082 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:53.083 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T06:45:53.121 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:45:53.155 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T06:45:53.183 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T06:45:53.211 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T06:45:53.221 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T06:45:53.239 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T06:45:53.269 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T06:45:53.421 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T06:45:53.437 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T06:45:53.458 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T06:45:53.479 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:45:53.486 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:53.524 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:45:53.566 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T06:45:53.593 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T06:45:53.598 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T06:45:53.625 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T06:45:53.649 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:45:53.664 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:45:53.760 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package jq. 2024-09-16T06:45:53.776 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T06:45:53.835 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T06:45:53.882 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T06:45:53.910 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T06:45:53.948 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T06:45:54.054 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package socat. 2024-09-16T06:45:54.071 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T06:45:54.078 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T06:45:54.105 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T06:45:54.113 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:45:54.134 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T06:45:54.136 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:45:54.150 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T06:45:54.184 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:45:54.269 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T06:45:54.296 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T06:45:54.335 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T06:45:54.431 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T06:45:54.441 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T06:45:54.458 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T06:45:54.462 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T06:45:54.469 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T06:45:54.489 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T06:45:54.497 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:45:54.508 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:45:54.528 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:45:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T06:45:54.782 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T06:45:54.798 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T06:45:54.808 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T06:45:54.837 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-test. 2024-09-16T06:45:54.841 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:45:54.847 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:45:54.853 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../096-ceph-test_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:54.895 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:54.930 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T06:45:54.958 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T06:45:54.996 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:45:55.459 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T06:45:55.486 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T06:45:55.510 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package radosgw. 2024-09-16T06:45:55.516 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:45:55.537 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../107-radosgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:55.561 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package radosgw. 2024-09-16T06:45:55.575 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:55.590 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../107-radosgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:55.628 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:55.820 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T06:45:55.848 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T06:45:55.886 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:45:56.138 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T06:45:56.167 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T06:45:56.205 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:45:56.329 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T06:45:56.357 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:56.372 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T06:45:56.389 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:56.422 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:56.690 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:56.701 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:56.816 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T06:45:56.846 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T06:45:56.899 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:56.933 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:56.960 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:56.976 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package radosgw. 2024-09-16T06:45:57.004 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../107-radosgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:57.059 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:57.086 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:57.161 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T06:45:57.177 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:57.211 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:57.297 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T06:45:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T06:45:57.556 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T06:45:57.583 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T06:45:57.622 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T06:45:57.622 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:57.712 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T06:45:57.770 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T06:45:57.786 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:45:57.828 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:57.927 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T06:45:57.943 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T06:45:57.986 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:45:58.047 INFO:teuthology.orchestra.run.smithi107.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T06:45:58.098 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T06:45:58.110 INFO:teuthology.orchestra.run.smithi117.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T06:45:58.118 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:58.171 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:58.358 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:45:58.381 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T06:45:58.409 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T06:45:58.447 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T06:45:58.509 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:45:58.626 INFO:teuthology.orchestra.run.smithi107.stdout:Adding system user cephadm....done 2024-09-16T06:45:58.703 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user cephadm....done 2024-09-16T06:45:58.776 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T06:45:58.783 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:45:58.794 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T06:45:58.803 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T06:45:58.842 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T06:45:58.852 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:45:59.050 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:45:59.109 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T06:45:59.120 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:45:59.171 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T06:45:59.198 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T06:45:59.237 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:45:59.294 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:45:59.379 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:45:59.420 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:45:59.506 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:45:59.532 INFO:teuthology.orchestra.run.smithi115.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T06:45:59.636 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:45:59.641 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T06:45:59.669 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T06:45:59.670 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T06:45:59.699 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:45:59.748 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T06:45:59.930 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:45:59.999 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:46:00.044 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T06:46:00.056 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T06:46:00.071 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T06:46:00.100 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T06:46:00.110 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:46:00.366 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:46:00.379 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T06:46:00.393 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:46:00.395 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T06:46:00.438 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:46:00.537 INFO:teuthology.orchestra.run.smithi115.stdout:Adding system user cephadm....done 2024-09-16T06:46:00.693 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:46:00.726 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T06:46:00.779 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T06:46:00.961 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:46:00.994 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T06:46:01.038 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T06:46:01.125 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package radosgw. 2024-09-16T06:46:01.153 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../107-radosgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:46:01.192 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:01.229 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:46:01.262 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:46:01.314 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:46:01.363 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:46:01.532 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:46:01.549 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:46:01.631 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T06:46:01.783 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:46:01.797 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:46:01.907 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:46:01.920 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T06:46:01.947 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T06:46:01.986 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:02.033 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T06:46:02.042 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:02.048 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:02.273 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T06:46:02.299 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:46:02.301 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:46:02.352 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:46:02.425 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:46:02.427 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:46:02.516 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T06:46:02.617 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T06:46:02.676 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:46:02.687 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:46:02.720 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T06:46:02.802 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T06:46:02.839 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T06:46:02.875 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T06:46:02.980 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T06:46:03.170 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:03.189 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:03.208 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T06:46:03.272 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:46:03.290 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T06:46:03.295 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T06:46:03.532 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:46:03.657 INFO:teuthology.orchestra.run.smithi100.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T06:46:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:46:03.657 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:46:03.711 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:03.784 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:46:03.808 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:46:03.822 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:46:04.075 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:04.093 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:46:04.115 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:46:04.169 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user cephadm....done 2024-09-16T06:46:04.308 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T06:46:04.360 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:46:04.478 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:46:04.585 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T06:46:04.737 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:46:04.787 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:46:04.861 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T06:46:04.888 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T06:46:04.901 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:46:04.987 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T06:46:05.054 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:46:05.169 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:46:05.214 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:46:05.230 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T06:46:05.264 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:05.286 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:46:05.340 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:46:05.390 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T06:46:05.445 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:46:05.489 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T06:46:05.591 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T06:46:05.666 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:46:05.746 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T06:46:05.884 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T06:46:05.951 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:46:05.957 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T06:46:06.122 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T06:46:06.244 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:46:06.252 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T06:46:06.342 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T06:46:06.365 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:46:06.504 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T06:46:06.585 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:46:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:46:06.771 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T06:46:06.887 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:46:06.961 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:46:07.001 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:46:07.022 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T06:46:07.155 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:46:07.262 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:46:07.273 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T06:46:07.323 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:46:07.344 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:46:07.474 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:46:07.566 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:07.595 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:46:07.613 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:46:07.721 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:07.767 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T06:46:07.818 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T06:46:07.848 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:46:07.944 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T06:46:07.957 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:08.030 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:46:08.151 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T06:46:08.220 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T06:46:08.242 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:46:08.346 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T06:46:08.348 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:46:08.403 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:46:08.534 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:46:08.714 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:08.779 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:46:08.840 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T06:46:09.097 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:46:09.116 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T06:46:09.400 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T06:46:09.448 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:46:09.520 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:09.640 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:09.654 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T06:46:09.710 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T06:46:09.733 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:46:09.757 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T06:46:09.859 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:09.897 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:46:10.000 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:46:10.023 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:46:10.126 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:46:10.149 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:46:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:46:10.353 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T06:46:10.419 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:46:10.425 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:46:10.536 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:46:10.613 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T06:46:10.676 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T06:46:10.739 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T06:46:10.769 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:46:10.804 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T06:46:10.840 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T06:46:10.927 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:11.039 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:11.053 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:46:11.108 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T06:46:11.165 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:46:11.179 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:46:11.266 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:46:11.497 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:46:11.500 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T06:46:11.550 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:46:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T06:46:11.752 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T06:46:11.810 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T06:46:11.982 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T06:46:12.017 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:12.085 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T06:46:12.111 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T06:46:12.168 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T06:46:12.233 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:46:12.353 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:46:12.379 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T06:46:12.428 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:46:12.493 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T06:46:12.553 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:46:12.604 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T06:46:12.679 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:46:12.752 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:46:12.797 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T06:46:12.864 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:46:12.870 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T06:46:12.973 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T06:46:12.981 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:46:13.040 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T06:46:13.129 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:46:13.150 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:46:13.248 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T06:46:13.249 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:46:13.350 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:46:13.434 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T06:46:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:46:13.516 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:13.627 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:46:13.642 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:46:13.743 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:46:13.760 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T06:46:13.807 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:46:13.927 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:46:14.058 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:14.061 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:46:14.179 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:14.329 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T06:46:14.524 INFO:teuthology.orchestra.run.smithi107.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T06:46:14.555 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T06:46:14.613 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T06:46:14.771 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:46:14.781 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:46:14.872 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:46:14.889 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:46:14.891 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:14.907 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:46:15.025 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T06:46:15.149 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T06:46:15.232 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:46:15.283 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:46:15.318 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T06:46:15.444 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T06:46:15.452 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:46:15.508 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T06:46:15.534 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T06:46:15.559 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T06:46:15.570 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T06:46:15.759 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:15.802 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:15.827 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:46:15.863 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T06:46:15.885 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:46:15.953 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:46:16.114 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T06:46:16.161 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:46:16.195 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:46:16.254 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:46:16.349 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:16.471 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:46:16.471 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:46:16.477 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T06:46:16.572 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:46:16.578 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T06:46:16.722 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T06:46:16.746 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:46:16.840 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T06:46:16.871 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T06:46:17.014 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T06:46:17.122 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T06:46:17.140 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T06:46:17.157 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T06:46:17.253 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T06:46:17.381 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T06:46:17.408 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T06:46:17.425 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:46:17.501 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:46:17.616 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T06:46:17.627 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:46:17.676 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:46:17.684 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:46:17.867 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T06:46:17.935 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:46:18.037 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:46:18.069 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T06:46:18.151 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T06:46:18.269 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T06:46:18.320 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T06:46:18.321 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T06:46:18.354 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:46:18.529 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T06:46:18.581 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:18.588 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:46:18.654 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T06:46:18.673 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:46:18.732 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:46:18.799 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:18.923 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:46:18.924 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:18.939 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T06:46:19.049 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.051 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.066 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:46:19.175 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.176 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.240 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T06:46:19.301 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.311 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.368 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:46:19.427 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.483 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T06:46:19.514 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T06:46:19.553 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.643 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T06:46:19.748 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T06:46:19.769 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T06:46:19.816 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:19.850 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T06:46:19.942 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.037 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T06:46:20.041 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.068 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.111 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T06:46:20.158 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.220 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T06:46:20.280 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:20.280 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:20.284 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.313 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:46:20.497 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:20.497 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:20.590 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:46:20.597 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T06:46:20.778 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.848 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T06:46:20.887 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.944 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:20.989 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:46:20.991 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T06:46:21.062 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:21.090 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T06:46:21.163 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:46:21.200 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T06:46:21.267 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:46:21.535 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T06:46:21.636 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:46:21.762 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:21.860 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T06:46:21.888 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.014 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.120 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T06:46:22.140 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.242 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.246 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T06:46:22.266 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.327 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.369 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.451 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.486 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T06:46:22.486 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.522 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T06:46:22.588 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.688 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.697 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.771 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.798 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T06:46:22.806 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:22.922 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:23.041 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T06:46:23.073 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:23.240 INFO:teuthology.orchestra.run.smithi107.stdout:Adding group ceph....done 2024-09-16T06:46:23.302 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:23.303 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:23.349 INFO:teuthology.orchestra.run.smithi117.stdout:Adding group ceph....done 2024-09-16T06:46:23.467 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T06:46:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:Adding system user ceph....done 2024-09-16T06:46:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:Setting system user ceph properties....done 2024-09-16T06:46:23.694 INFO:teuthology.orchestra.run.smithi107.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T06:46:23.701 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user ceph....done 2024-09-16T06:46:23.752 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T06:46:23.799 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:23.828 INFO:teuthology.orchestra.run.smithi117.stdout:Setting system user ceph properties....done 2024-09-16T06:46:23.832 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T06:46:23.846 INFO:teuthology.orchestra.run.smithi117.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T06:46:23.934 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:23.981 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T06:46:24.071 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T06:46:24.077 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:46:24.171 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T06:46:24.188 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.281 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T06:46:24.289 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.390 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.492 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.634 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.648 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.740 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.774 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:24.846 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T06:46:24.867 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.122 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.229 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:25.229 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:25.248 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.327 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:25.328 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:25.347 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.374 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.587 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:25.587 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T06:46:25.718 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.740 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.777 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:25.875 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.001 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.068 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T06:46:26.119 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T06:46:26.152 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.169 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.270 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T06:46:26.570 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.671 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:26.793 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:26.794 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:26.803 INFO:teuthology.orchestra.run.smithi115.stdout:Adding group ceph....done 2024-09-16T06:46:26.898 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:26.899 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:27.180 INFO:teuthology.orchestra.run.smithi115.stdout:Adding system user ceph....done 2024-09-16T06:46:27.264 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.347 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.390 INFO:teuthology.orchestra.run.smithi115.stdout:Setting system user ceph properties....done 2024-09-16T06:46:27.391 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.412 INFO:teuthology.orchestra.run.smithi115.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T06:46:27.466 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:27.466 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:27.517 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.545 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T06:46:27.566 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:27.566 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:27.668 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.794 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.853 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T06:46:27.915 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:27.920 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.049 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.187 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:28.187 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:28.320 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.353 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:28.353 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:28.405 INFO:teuthology.orchestra.run.smithi100.stdout:Adding group ceph....done 2024-09-16T06:46:28.446 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.683 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.757 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user ceph....done 2024-09-16T06:46:28.809 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.861 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:28.925 INFO:teuthology.orchestra.run.smithi100.stdout:Setting system user ceph properties....done 2024-09-16T06:46:28.941 INFO:teuthology.orchestra.run.smithi100.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T06:46:28.960 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.063 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.063 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:29.063 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:29.076 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T06:46:29.174 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:29.174 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:29.324 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.409 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T06:46:29.514 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.624 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:29.625 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:29.627 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.779 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.847 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T06:46:29.880 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:29.914 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.014 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.040 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.113 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.264 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.325 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.359 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T06:46:30.407 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.524 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:30.524 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T06:46:30.541 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:30.541 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:30.958 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:30.976 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:31.035 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:31.288 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:31.289 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:31.300 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T06:46:31.313 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T06:46:31.594 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T06:46:31.770 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:31.776 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:31.993 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:31.993 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T06:46:32.039 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:32.040 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:32.080 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T06:46:32.086 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T06:46:32.093 DEBUG:teuthology.parallel:result is None 2024-09-16T06:46:32.478 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:32.504 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:32.565 INFO:teuthology.orchestra.run.smithi107.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T06:46:32.571 DEBUG:teuthology.parallel:result is None 2024-09-16T06:46:32.632 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:32.684 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:32.685 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T06:46:32.790 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:33.021 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:33.021 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:33.188 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:33.473 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:33.473 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T06:46:33.483 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:33.625 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:33.776 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:33.953 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:33.956 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:34.082 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:34.233 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:34.322 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T06:46:34.463 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:34.463 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T06:46:34.960 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:35.104 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:35.204 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:35.346 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T06:46:35.548 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T06:46:35.683 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T06:46:36.016 INFO:teuthology.orchestra.run.smithi115.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T06:46:36.022 DEBUG:teuthology.parallel:result is None 2024-09-16T06:46:36.775 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T06:46:37.291 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T06:46:37.297 DEBUG:teuthology.parallel:result is None 2024-09-16T06:46:37.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:46:37.440 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-query -W -f '${Version}' ceph 2024-09-16T06:46:37.459 INFO:teuthology.orchestra.run.smithi100.stdout:18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.460 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.461 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1jammy is installed. 2024-09-16T06:46:37.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:46:37.606 DEBUG:teuthology.orchestra.run.smithi107:> dpkg-query -W -f '${Version}' ceph 2024-09-16T06:46:37.625 INFO:teuthology.orchestra.run.smithi107.stdout:18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.625 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.625 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1jammy is installed. 2024-09-16T06:46:37.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:46:37.761 DEBUG:teuthology.orchestra.run.smithi115:> dpkg-query -W -f '${Version}' ceph 2024-09-16T06:46:37.780 INFO:teuthology.orchestra.run.smithi115.stdout:18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.780 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.780 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1jammy is installed. 2024-09-16T06:46:37.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T06:46:37.915 DEBUG:teuthology.orchestra.run.smithi117:> dpkg-query -W -f '${Version}' ceph 2024-09-16T06:46:37.934 INFO:teuthology.orchestra.run.smithi117.stdout:18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.934 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T06:46:37.934 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1jammy is installed. 2024-09-16T06:46:37.936 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T06:46:37.937 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:37.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T06:46:37.952 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:37.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T06:46:37.966 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:37.966 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T06:46:37.981 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:37.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T06:46:37.996 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T06:46:37.996 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:37.996 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T06:46:38.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T06:46:38.061 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:38.061 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T06:46:38.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T06:46:38.129 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:38.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T06:46:38.144 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T06:46:38.200 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:38.201 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T06:46:38.214 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T06:46:38.268 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T06:46:38.269 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:38.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T06:46:38.282 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T06:46:38.336 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:38.336 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T06:46:38.349 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T06:46:38.402 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:38.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T06:46:38.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T06:46:38.469 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:38.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T06:46:38.482 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T06:46:38.536 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T06:46:38.536 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:38.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T06:46:38.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T06:46:38.604 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:38.605 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T06:46:38.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T06:46:38.673 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:38.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T06:46:38.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T06:46:38.741 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:38.741 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T06:46:38.754 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T06:46:38.808 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T06:46:38.886 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T06:46:38.886 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-09-16T06:46:38.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2024-09-16T06:46:38.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-09-16T06:46:38.893 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2024-09-16T06:46:38.906 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T06:46:38.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T06:46:38.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T06:46:38.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T06:46:38.950 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T06:46:38.962 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T06:46:38.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T06:46:38.995 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T06:46:38.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T06:46:39.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T06:46:39.030 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T06:46:39.030 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T06:46:39.030 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576', 'branch': 'reef', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906576', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 931, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 1390, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'sleep_before_teardown': 0, 'subset': '5319/100000', 'suite': 'rados', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6', 'targets': {'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD5WngGd5C+n1xBEgR/2fLngmLEmV0kcKYKLwoTCspgnGubGFf9xwLlvHsJDxr2o953DxJKfwcIpulEOuZxr8yE=', 'smithi107.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGUu56rlMc0r3OgPFOfwbcLd/UNst1IOzCJ9yR7NJX71l+Py0F5QG762RDSs2gZdXCdmvlHgZN9deLQM0rbYEk=', 'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7FDZqLGTRvt5hJym7QalftMihip+TP1QjSm5eydH2p5Vzm/5ac6f9kwBbEhqrOn45zoKAuGR73MOfztco2wzM=', 'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIklEbjPEULrsj2slKPj7JWlq8NAxwUzzeii0j4N99r7c6lEMNVI5oNHF7YYFUdKQBgQwRVXcKM/W216AAwZloU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'cluster': 'ceph'}}, {'thrashosds': {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_22:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T06:46:39.031 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T06:46:39.055 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T06:46:39.057 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T06:46:39.062 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T06:46:39.067 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T06:46:39.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T06:46:39.105 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T06:46:39.110 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T06:46:39.122 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:39.122 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:46:39.162 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T06:46:39.162 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-16T06:46:39.212 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 06:41:12.197722583 +0000 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 06:41:11.917725085 +0000 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 06:41:11.917725085 +0000 2024-09-16T06:46:39.213 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-16T06:46:39.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T06:46:39.268 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T06:46:39.269 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T06:46:39.269 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000429815 s, 1.2 MB/s 2024-09-16T06:46:39.270 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T06:46:39.320 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 06:41:12.689718189 +0000 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 06:41:12.445720368 +0000 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 06:41:12.445720368 +0000 2024-09-16T06:46:39.366 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-16T06:46:39.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T06:46:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T06:46:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T06:46:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000364765 s, 1.4 MB/s 2024-09-16T06:46:39.421 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T06:46:39.471 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 06:41:13.193713688 +0000 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 06:41:12.929716045 +0000 2024-09-16T06:46:39.518 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 06:41:12.929716045 +0000 2024-09-16T06:46:39.519 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-16T06:46:39.519 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T06:46:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T06:46:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T06:46:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000323232 s, 1.6 MB/s 2024-09-16T06:46:39.573 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T06:46:39.619 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 06:41:13.709709079 +0000 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 06:41:13.441711473 +0000 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 06:41:13.441711473 +0000 2024-09-16T06:46:39.666 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-16T06:46:39.666 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T06:46:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T06:46:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T06:46:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000330451 s, 1.5 MB/s 2024-09-16T06:46:39.722 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T06:46:39.771 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:39.771 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:39.771 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:46:39.777 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T06:46:39.777 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-16T06:46:39.823 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T06:46:39.823 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:39.823 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-09-16T06:46:39.823 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:39.823 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-16 06:41:10.097164369 +0000 2024-09-16T06:46:39.823 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-16 06:41:09.841166655 +0000 2024-09-16T06:46:39.824 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-16 06:41:09.841166655 +0000 2024-09-16T06:46:39.824 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-16T06:46:39.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T06:46:39.877 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-16T06:46:39.877 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-16T06:46:39.877 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000410217 s, 1.2 MB/s 2024-09-16T06:46:39.878 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T06:46:39.928 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-16T06:46:39.974 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T06:46:39.974 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:39.975 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-09-16T06:46:39.975 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:39.975 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-16 06:41:10.333162261 +0000 2024-09-16T06:46:39.975 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-16 06:41:10.329162297 +0000 2024-09-16T06:46:39.975 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-16 06:41:10.329162297 +0000 2024-09-16T06:46:39.975 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-16T06:46:39.975 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T06:46:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-16T06:46:40.029 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-16T06:46:40.029 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000314243 s, 1.6 MB/s 2024-09-16T06:46:40.030 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T06:46:40.080 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-16T06:46:40.126 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T06:46:40.126 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:40.126 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-09-16T06:46:40.126 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:40.127 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-16 06:41:11.065155723 +0000 2024-09-16T06:46:40.127 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-16 06:41:10.801158081 +0000 2024-09-16T06:46:40.127 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-16 06:41:10.801158081 +0000 2024-09-16T06:46:40.127 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-16T06:46:40.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T06:46:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-16T06:46:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-16T06:46:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000366426 s, 1.4 MB/s 2024-09-16T06:46:40.182 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T06:46:40.232 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-16 06:41:11.293153687 +0000 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-16 06:41:11.289153722 +0000 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-16 06:41:11.289153722 +0000 2024-09-16T06:46:40.279 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-16T06:46:40.280 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T06:46:40.332 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-16T06:46:40.332 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-16T06:46:40.332 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000299605 s, 1.7 MB/s 2024-09-16T06:46:40.333 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T06:46:40.385 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:40.385 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:40.385 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:46:40.391 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T06:46:40.391 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-16T06:46:40.437 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T06:46:40.437 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:40.438 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-09-16T06:46:40.438 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:40.438 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-16 06:41:12.282730141 +0000 2024-09-16T06:46:40.438 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-16 06:41:12.034732356 +0000 2024-09-16T06:46:40.438 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-16 06:41:12.034732356 +0000 2024-09-16T06:46:40.438 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-16T06:46:40.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T06:46:40.492 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-16T06:46:40.493 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-16T06:46:40.493 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00045027 s, 1.1 MB/s 2024-09-16T06:46:40.494 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T06:46:40.546 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-16 06:41:12.790725604 +0000 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-16 06:41:12.526727962 +0000 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-16 06:41:12.526727962 +0000 2024-09-16T06:46:40.594 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-16T06:46:40.595 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T06:46:40.647 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-16T06:46:40.647 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-16T06:46:40.647 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000332183 s, 1.5 MB/s 2024-09-16T06:46:40.648 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T06:46:40.699 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-16T06:46:40.749 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T06:46:40.749 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:40.749 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 913 Links: 1 2024-09-16T06:46:40.749 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:40.750 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-16 06:41:13.038723389 +0000 2024-09-16T06:46:40.750 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-16 06:41:13.034723425 +0000 2024-09-16T06:46:40.750 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-16 06:41:13.034723425 +0000 2024-09-16T06:46:40.750 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-16T06:46:40.750 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T06:46:40.803 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-16T06:46:40.804 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-16T06:46:40.804 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000438484 s, 1.2 MB/s 2024-09-16T06:46:40.805 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T06:46:40.854 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-16T06:46:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T06:46:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:40.901 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-16T06:46:40.901 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:40.901 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-16 06:41:13.494719316 +0000 2024-09-16T06:46:40.901 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-16 06:41:13.490719352 +0000 2024-09-16T06:46:40.902 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-16 06:41:13.490719352 +0000 2024-09-16T06:46:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-16T06:46:40.902 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T06:46:40.955 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-16T06:46:40.956 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-16T06:46:40.956 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000382586 s, 1.3 MB/s 2024-09-16T06:46:40.957 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T06:46:41.006 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:41.007 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:41.007 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T06:46:41.012 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T06:46:41.013 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-16T06:46:41.061 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-16 06:41:13.800939711 +0000 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-16 06:41:13.556941891 +0000 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-16 06:41:13.556941891 +0000 2024-09-16T06:46:41.062 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-09-16T06:46:41.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T06:46:41.116 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-16T06:46:41.116 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-16T06:46:41.116 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000447763 s, 1.1 MB/s 2024-09-16T06:46:41.117 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T06:46:41.167 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-16 06:41:14.336934924 +0000 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-16 06:41:14.064937353 +0000 2024-09-16T06:46:41.213 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-16 06:41:14.064937353 +0000 2024-09-16T06:46:41.214 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-09-16T06:46:41.214 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T06:46:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-16T06:46:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-16T06:46:41.267 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000346176 s, 1.5 MB/s 2024-09-16T06:46:41.268 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T06:46:41.318 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-16T06:46:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-16 06:41:14.840930422 +0000 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-16 06:41:14.584932709 +0000 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-16 06:41:14.584932709 +0000 2024-09-16T06:46:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-09-16T06:46:41.366 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T06:46:41.419 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-16T06:46:41.420 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-16T06:46:41.420 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000419815 s, 1.2 MB/s 2024-09-16T06:46:41.421 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T06:46:41.470 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-16 06:41:15.328926063 +0000 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-16 06:41:15.084928242 +0000 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-16 06:41:15.084928242 +0000 2024-09-16T06:46:41.517 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-09-16T06:46:41.518 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T06:46:41.571 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-16T06:46:41.571 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-16T06:46:41.571 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000337752 s, 1.5 MB/s 2024-09-16T06:46:41.572 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T06:46:41.622 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:41.622 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-16T06:46:41.622 INFO:tasks.ceph:Generating config... 2024-09-16T06:46:41.623 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T06:46:41.624 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.1 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-16T06:46:41.625 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-16T06:46:41.626 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] ms inject delay max = 1 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] ms inject delay type = osd 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] osd_recovery_max_active = 10 2024-09-16T06:46:41.626 INFO:tasks.ceph:[global] osd_recovery_max_single_start = 10 2024-09-16T06:46:41.626 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T06:46:41.626 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T06:46:41.626 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T06:46:41.626 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T06:46:41.626 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T06:46:41.626 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-16T06:46:41.627 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T06:46:41.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T06:46:41.662 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T06:46:41.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T06:46:41.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T06:46:41.805 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.100'), ('mon.b', '172.21.15.107'), ('mon.c', '172.21.15.115')] 2024-09-16T06:46:41.805 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.100,172.21.15.107,172.21.15.115', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-16T06:46:41.806 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:41.806 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T06:46:41.858 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.100 --add b 172.21.15.107 --add c 172.21.15.115 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: generated fsid b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = pacific 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 0 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:fsid b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:last_changed 2024-09-16T06:46:41.945739+0000 2024-09-16T06:46:41.943 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-09-16T06:46:41.945739+0000 2024-09-16T06:46:41.944 INFO:teuthology.orchestra.run.smithi100.stdout:min_mon_release 16 (pacific) 2024-09-16T06:46:41.944 INFO:teuthology.orchestra.run.smithi100.stdout:election_strategy: 3 2024-09-16T06:46:41.944 INFO:teuthology.orchestra.run.smithi100.stdout:0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-09-16T06:46:41.944 INFO:teuthology.orchestra.run.smithi100.stdout:1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-16T06:46:41.944 INFO:teuthology.orchestra.run.smithi100.stdout:2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.c 2024-09-16T06:46:41.944 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T06:46:41.945 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T06:46:41.994 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b2976256-637c-4e28-94f6-99f20017bc5e... 2024-09-16T06:46:41.995 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T06:46:42.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T06:46:42.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T06:46:42.044 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T06:46:42.066 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-09-16T06:46:42.066 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-09-16T06:46:42.066 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T06:46:42.066 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-09-16T06:46:42.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-09-16T06:46:42.068 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-09-16T06:46:42.069 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.100,172.21.15.107,172.21.15.115" 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: mon client directed command retry = 5 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: mon election default strategy = 3 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay max = 1 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay probability = 0.005 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay type = osd 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject internal delays = 0.002 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject socket failures = 2500 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: osd_recovery_max_active = 10 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: osd_recovery_max_single_start = 10 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-09-16T06:46:42.070 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 120 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-09-16T06:46:42.071 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-09-16T06:46:42.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.073 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-09-16T06:46:42.073 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-09-16T06:46:42.073 INFO:teuthology.orchestra.run.smithi100.stdout: debug monc = 20 2024-09-16T06:46:42.073 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-09-16T06:46:42.073 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-09-16T06:46:42.073 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore zero block detection = True 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 20 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 20 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 10 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: osd blocked scrub grace period = 3600 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug reject backfill probability = 0.1 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify cached snaps = True 2024-09-16T06:46:42.074 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify missing on start = True 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd max backfills = 3 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd max markdown count = 1000 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock override recovery settings = True 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd memory target = 1610612736 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd op queue = debug_random 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd op queue cut off = debug_random 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub min interval = 60 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: osd shutdown pgref assert = True 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-09-16T06:46:42.075 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-09-16T06:46:42.076 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T06:46:42.077 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: mon scrub interval = 300 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-09-16T06:46:42.078 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-09-16T06:46:42.079 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T06:46:42.079 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T06:46:42.079 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-09-16T06:46:42.079 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-09-16T06:46:42.079 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.080 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.081 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.082 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.100,172.21.15.107,172.21.15.115" 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: mon client directed command retry = 5 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: mon election default strategy = 3 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay max = 1 2024-09-16T06:46:42.083 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay probability = 0.005 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay type = osd 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject internal delays = 0.002 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject socket failures = 2500 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: osd_recovery_max_active = 10 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: osd_recovery_max_single_start = 10 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 120 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-09-16T06:46:42.084 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-09-16T06:46:42.085 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: debug monc = 20 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-09-16T06:46:42.086 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore zero block detection = True 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 20 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 20 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 10 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: osd blocked scrub grace period = 3600 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug reject backfill probability = 0.1 2024-09-16T06:46:42.087 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify cached snaps = True 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify missing on start = True 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd max backfills = 3 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd max markdown count = 1000 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock override recovery settings = True 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd memory target = 1610612736 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd op queue = debug_random 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd op queue cut off = debug_random 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub min interval = 60 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: osd shutdown pgref assert = True 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.088 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-09-16T06:46:42.089 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T06:46:42.090 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: mon scrub interval = 300 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T06:46:42.091 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-09-16T06:46:42.092 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.093 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T06:46:42.094 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.100,172.21.15.107,172.21.15.115" 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: mon client directed command retry = 5 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: mon election default strategy = 3 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject delay max = 1 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject delay probability = 0.005 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject delay type = osd 2024-09-16T06:46:42.095 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject internal delays = 0.002 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject socket failures = 2500 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd_recovery_max_active = 10 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd_recovery_max_single_start = 10 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 120 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-09-16T06:46:42.096 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-09-16T06:46:42.097 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: debug monc = 20 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: bdev async discard = True 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: bdev enable discard = True 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore zero block detection = True 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 20 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 20 2024-09-16T06:46:42.098 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 10 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd blocked scrub grace period = 3600 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug reject backfill probability = 0.1 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify cached snaps = True 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify missing on start = True 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd max backfills = 3 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd max markdown count = 1000 2024-09-16T06:46:42.099 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock override recovery settings = True 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: osd memory target = 1610612736 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue = debug_random 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue cut off = debug_random 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub min interval = 60 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: osd shutdown pgref assert = True 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-16T06:46:42.100 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-09-16T06:46:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: mon scrub interval = 300 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-09-16T06:46:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2024-09-16T06:46:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2024-09-16T06:46:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.100,172.21.15.107,172.21.15.115" 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: mon client directed command retry = 5 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: mon election default strategy = 3 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject delay max = 1 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject delay probability = 0.005 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject delay type = osd 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject internal delays = 0.002 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject socket failures = 2500 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: osd_recovery_max_active = 10 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: osd_recovery_max_single_start = 10 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2024-09-16T06:46:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 120 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2024-09-16T06:46:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: debug monc = 20 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: bdev async discard = True 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: bdev enable discard = True 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore zero block detection = True 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 20 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 20 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 10 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T06:46:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd blocked scrub grace period = 3600 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug reject backfill probability = 0.1 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify cached snaps = True 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify missing on start = True 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd max backfills = 3 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd max markdown count = 1000 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock override recovery settings = True 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd memory target = 1610612736 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd op queue = debug_random 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd op queue cut off = debug_random 2024-09-16T06:46:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub min interval = 60 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: osd shutdown pgref assert = True 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-16T06:46:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T06:46:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: mon scrub interval = 300 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T06:46:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T06:46:42.115 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2024-09-16T06:46:42.115 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2024-09-16T06:46:42.115 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2024-09-16T06:46:42.117 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T06:46:42.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T06:46:42.206 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T06:46:42.207 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:42.207 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T06:46:42.212 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:42.212 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T06:46:42.257 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:46:42.258 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:42.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T06:46:42.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T06:46:42.321 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:46:42.321 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:42.370 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:46:42.370 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:42.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T06:46:42.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T06:46:42.394 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:46:42.394 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:42.442 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:46:42.443 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:42.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T06:46:42.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T06:46:42.466 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:46:42.466 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:42.513 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:46:42.513 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:42.513 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T06:46:42.513 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T06:46:42.537 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:46:42.537 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:42.585 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T06:46:42.585 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T06:46:42.585 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T06:46:42.631 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T06:46:42.634 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T06:46:42.677 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T06:46:42.686 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T06:46:42.686 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T06:46:42.686 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T06:46:42.761 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T06:46:42.772 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T06:46:42.773 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-16T06:46:42.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T06:46:42.787 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:42.787 INFO:tasks.ceph:role: osd.0 2024-09-16T06:46:42.787 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:46:42.787 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:42.850 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:42.851 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:42.851 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:43.297 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T06:46:43.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T06:46:43.299 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T06:46:43.393 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T06:46:43.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:46:43.404 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:46:43.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T06:46:43.457 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:43.458 INFO:tasks.ceph:role: osd.4 2024-09-16T06:46:43.458 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:46:43.458 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T06:46:43.523 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:43.523 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:43.523 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:43.523 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:43.523 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:43.523 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:43.524 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:43.524 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:43.524 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:43.524 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:43.989 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T06:46:43.991 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T06:46:43.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-16T06:46:44.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T06:46:44.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:46:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:46:44.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-16T06:46:44.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:44.217 INFO:tasks.ceph:role: osd.8 2024-09-16T06:46:44.218 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:46:44.218 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:44.282 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:44.744 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T06:46:44.746 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T06:46:44.746 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-09-16T06:46:44.886 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-16T06:46:44.897 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:46:44.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:46:44.899 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-09-16T06:46:44.953 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:44.953 INFO:tasks.ceph:role: osd.12 2024-09-16T06:46:44.954 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:46:44.954 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T06:46:45.021 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:45.021 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:45.022 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:45.442 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T06:46:45.444 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T06:46:45.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-09-16T06:46:45.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-09-16T06:46:45.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:46:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:46:45.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:45.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:45.649+0000 7f6059ba47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T06:46:45.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:45.649+0000 7f6059ba47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T06:46:45.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:45.649+0000 7f6059ba47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T06:46:45.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:45.649+0000 7f6059ba47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T06:46:45.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:45.649+0000 7f6059ba47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T06:46:45.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:45.653+0000 7f6059ba47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T06:46:48.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T06:46:48.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:48.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:48.737+0000 7f55e33a37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T06:46:48.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:48.737+0000 7f55e33a37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T06:46:48.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:48.737+0000 7f55e33a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T06:46:48.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:48.737+0000 7f55e33a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T06:46:48.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:48.737+0000 7f55e33a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T06:46:48.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:48.737+0000 7f55e33a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T06:46:51.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T06:46:51.795 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:51.877+0000 7f6395bf87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-16T06:46:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:51.877+0000 7f6395bf87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-16T06:46:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:51.877+0000 7f6395bf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-16T06:46:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:51.877+0000 7f6395bf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-16T06:46:51.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:51.877+0000 7f6395bf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-16T06:46:51.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:51.877+0000 7f6395bf87c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-16T06:46:54.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-16T06:46:54.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:46:54.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:54.985+0000 7fedd5a8f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-09-16T06:46:54.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:54.985+0000 7fedd5a8f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-09-16T06:46:54.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:54.985+0000 7fedd5a8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-09-16T06:46:54.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:54.985+0000 7fedd5a8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-09-16T06:46:54.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:54.985+0000 7fedd5a8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-09-16T06:46:54.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:46:54.985+0000 7fedd5a8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-09-16T06:46:57.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-09-16T06:46:58.023 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T06:46:58.038 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:58.038 INFO:tasks.ceph:role: osd.1 2024-09-16T06:46:58.038 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:46:58.039 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:58.102 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:58.553 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-16T06:46:58.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-16T06:46:58.555 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-16T06:46:58.655 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T06:46:58.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:46:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:46:58.665 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T06:46:58.715 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:58.715 INFO:tasks.ceph:role: osd.5 2024-09-16T06:46:58.715 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:46:58.715 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:58.778 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:59.239 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-16T06:46:59.241 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-16T06:46:59.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T06:46:59.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T06:46:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:46:59.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:46:59.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-16T06:46:59.434 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-16T06:46:59.434 INFO:tasks.ceph:role: osd.9 2024-09-16T06:46:59.434 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:46:59.434 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T06:46:59.498 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:46:59.499 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:46:59.499 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:46:59.499 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:46:59.499 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:46:59.499 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:46:59.500 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:46:59.500 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:46:59.500 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:46:59.500 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:46:59.987 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-16T06:46:59.990 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-16T06:46:59.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-09-16T06:47:00.135 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-16T06:47:00.146 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:00.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:00.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-09-16T06:47:00.202 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:00.202 INFO:tasks.ceph:role: osd.13 2024-09-16T06:47:00.202 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:47:00.202 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:00.265 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:00.694 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-16T06:47:00.697 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-16T06:47:00.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-09-16T06:47:00.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-09-16T06:47:00.796 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:00.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:00.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:00.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:00.914+0000 7f88badae7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T06:47:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:00.914+0000 7f88badae7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T06:47:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:00.914+0000 7f88badae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T06:47:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:00.914+0000 7f88badae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T06:47:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:00.914+0000 7f88badae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T06:47:00.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:00.922+0000 7f88badae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T06:47:03.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T06:47:03.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:04.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:04.018+0000 7ff5b3f757c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T06:47:04.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:04.018+0000 7ff5b3f757c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T06:47:04.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:04.018+0000 7ff5b3f757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T06:47:04.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:04.018+0000 7ff5b3f757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T06:47:04.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:04.018+0000 7ff5b3f757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T06:47:04.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:04.018+0000 7ff5b3f757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T06:47:07.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T06:47:07.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:07.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:07.162+0000 7fbd287c87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-16T06:47:07.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:07.166+0000 7fbd287c87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-16T06:47:07.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:07.166+0000 7fbd287c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-16T06:47:07.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:07.166+0000 7fbd287c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-16T06:47:07.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:07.166+0000 7fbd287c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-16T06:47:07.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:07.166+0000 7fbd287c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-16T06:47:10.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-16T06:47:10.187 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:10.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:10.270+0000 7fe1804e17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-09-16T06:47:10.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:10.270+0000 7fe1804e17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-09-16T06:47:10.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:10.270+0000 7fe1804e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-09-16T06:47:10.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:10.270+0000 7fe1804e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-09-16T06:47:10.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:10.270+0000 7fe1804e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-09-16T06:47:10.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-16T06:47:10.270+0000 7fe1804e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-09-16T06:47:13.228 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-09-16T06:47:13.283 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T06:47:13.298 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:13.298 INFO:tasks.ceph:role: osd.2 2024-09-16T06:47:13.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:47:13.298 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T06:47:13.360 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:13.361 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:13.826 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-16T06:47:13.828 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-16T06:47:13.828 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-16T06:47:13.967 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T06:47:13.978 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:13.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:13.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T06:47:14.032 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:14.033 INFO:tasks.ceph:role: osd.6 2024-09-16T06:47:14.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:47:14.033 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:14.097 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:14.522 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-16T06:47:14.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-16T06:47:14.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-09-16T06:47:14.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T06:47:14.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:14.637 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:14.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-16T06:47:14.692 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:14.692 INFO:tasks.ceph:role: osd.10 2024-09-16T06:47:14.692 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:47:14.692 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T06:47:14.757 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:14.757 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:14.757 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:14.757 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:14.757 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:14.757 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:14.758 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:14.758 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:14.758 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:14.758 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:15.228 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-16T06:47:15.230 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-16T06:47:15.230 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-16T06:47:15.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-16T06:47:15.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:15.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-09-16T06:47:15.415 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:15.415 INFO:tasks.ceph:role: osd.14 2024-09-16T06:47:15.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:47:15.415 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T06:47:15.477 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:15.478 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:15.478 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:15.479 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:15.479 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:15.939 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-16T06:47:15.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-16T06:47:15.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-09-16T06:47:16.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-09-16T06:47:16.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:16.105 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:16.105 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:16.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:16.220+0000 7f25320867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T06:47:16.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:16.224+0000 7f25320867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T06:47:16.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:16.224+0000 7f25320867c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T06:47:16.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:16.224+0000 7f25320867c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T06:47:16.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:16.224+0000 7f25320867c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T06:47:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:16.232+0000 7f25320867c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T06:47:19.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T06:47:19.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:19.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:19.308+0000 7f3dabab27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T06:47:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:19.308+0000 7f3dabab27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T06:47:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:19.308+0000 7f3dabab27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T06:47:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:19.308+0000 7f3dabab27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T06:47:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:19.308+0000 7f3dabab27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T06:47:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:19.308+0000 7f3dabab27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T06:47:22.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T06:47:22.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:22.428+0000 7f826f7427c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-16T06:47:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:22.428+0000 7f826f7427c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-16T06:47:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:22.428+0000 7f826f7427c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-16T06:47:22.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:22.428+0000 7f826f7427c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-16T06:47:22.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:22.428+0000 7f826f7427c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-16T06:47:22.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:22.428+0000 7f826f7427c0 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-16T06:47:25.392 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-16T06:47:25.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:25.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:25.524+0000 7fbb15f507c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-09-16T06:47:25.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:25.524+0000 7fbb15f507c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-09-16T06:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:25.524+0000 7fbb15f507c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-09-16T06:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:25.524+0000 7fbb15f507c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-09-16T06:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:25.524+0000 7fbb15f507c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-09-16T06:47:25.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-16T06:47:25.524+0000 7fbb15f507c0 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-09-16T06:47:28.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-09-16T06:47:28.581 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T06:47:28.596 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:28.596 INFO:tasks.ceph:role: osd.3 2024-09-16T06:47:28.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:47:28.596 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T06:47:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:29.118 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-09-16T06:47:29.120 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-09-16T06:47:29.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-16T06:47:29.224 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T06:47:29.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:29.234 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:29.234 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T06:47:29.286 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:29.286 INFO:tasks.ceph:role: osd.7 2024-09-16T06:47:29.286 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:47:29.286 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:29.349 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:29.350 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:29.350 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:29.350 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-09-16T06:47:29.816 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-09-16T06:47:29.816 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-09-16T06:47:29.946 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T06:47:29.957 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:29.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:29.958 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-16T06:47:30.012 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:30.012 INFO:tasks.ceph:role: osd.11 2024-09-16T06:47:30.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:47:30.012 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T06:47:30.074 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:30.075 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:30.560 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-09-16T06:47:30.562 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-09-16T06:47:30.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-09-16T06:47:30.711 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-16T06:47:30.722 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:30.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:30.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-09-16T06:47:30.780 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-16T06:47:30.780 INFO:tasks.ceph:role: osd.15 2024-09-16T06:47:30.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:47:30.780 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T06:47:30.845 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T06:47:30.845 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T06:47:30.845 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T06:47:30.845 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T06:47:30.845 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T06:47:30.846 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-09-16T06:47:30.846 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T06:47:30.846 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T06:47:30.846 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T06:47:30.846 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T06:47:31.265 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-09-16T06:47:31.266 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-09-16T06:47:31.267 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-09-16T06:47:31.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-09-16T06:47:31.381 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T06:47:31.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:47:31.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:31.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:31.494+0000 7f07224da7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T06:47:31.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:31.494+0000 7f07224da7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T06:47:31.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:31.494+0000 7f07224da7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T06:47:31.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:31.494+0000 7f07224da7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T06:47:31.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:31.494+0000 7f07224da7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T06:47:31.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:31.498+0000 7f07224da7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T06:47:34.464 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T06:47:34.478 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:34.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:34.562+0000 7fa1ccb707c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T06:47:34.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:34.562+0000 7fa1ccb707c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T06:47:34.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:34.562+0000 7fa1ccb707c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T06:47:34.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:34.562+0000 7fa1ccb707c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T06:47:34.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:34.562+0000 7fa1ccb707c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T06:47:34.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:34.562+0000 7fa1ccb707c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T06:47:37.528 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T06:47:37.585 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:37.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:37.666+0000 7fa9f82c37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-16T06:47:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:37.666+0000 7fa9f82c37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-16T06:47:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:37.666+0000 7fa9f82c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-16T06:47:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:37.666+0000 7fa9f82c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-16T06:47:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:37.666+0000 7fa9f82c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-16T06:47:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:37.670+0000 7fa9f82c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-16T06:47:40.662 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-16T06:47:40.718 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:40.798+0000 7fd06500e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-09-16T06:47:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:40.802+0000 7fd06500e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-09-16T06:47:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:40.802+0000 7fd06500e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-09-16T06:47:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:40.802+0000 7fd06500e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-09-16T06:47:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:40.802+0000 7fd06500e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-09-16T06:47:40.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:40.802+0000 7fd06500e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-09-16T06:47:43.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-09-16T06:47:43.845 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T06:47:43.845 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:43.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T06:47:43.860 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:43.860 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T06:47:43.916 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:43.916 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T06:47:43.971 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:43.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-16T06:47:44.024 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:44.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-09-16T06:47:44.080 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:44.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T06:47:44.098 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:44.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T06:47:44.154 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:44.154 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-16T06:47:44.210 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:44.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-09-16T06:47:44.264 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:44.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T06:47:44.279 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:44.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T06:47:44.331 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:44.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-16T06:47:44.383 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:44.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-09-16T06:47:44.439 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:44.439 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T06:47:44.453 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:44.453 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T06:47:44.512 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:44.512 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T06:47:44.567 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:44.567 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-16T06:47:44.620 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:44.620 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-09-16T06:47:44.676 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:44.676 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T06:47:44.725 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T06:47:44.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T06:47:44.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T06:47:44.729 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBS1OdmPJnJJRAA31BkN+jCB28u2yyIJXMl0w== 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBV1Odme2kOJxAAtPNbNXqWG8f7QMvxlFlcew== 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBY1Odm19stLBAADGiF4WSOtT243JUWTXD0Gg== 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.8] 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBb1Odmdm5+NBAAzxC4wAr88JVPPFaNa1wzFw== 2024-09-16T06:47:44.737 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.12] 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBe1Odmd1QDOxAAZ0wwJsF5txNQ5XhhH8lx2A== 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBk1Odmfb7cNhAAFSGmqvFMAQm+kp1tYH0ZHg== 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBo1Odm8Dk4ARAAxH6mjclBN8As2xQZvyjzew== 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.9] 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBr1OdmnermCRAA1pTYeThGPNX7FFKrJPOoCA== 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.13] 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBu1Odmq6MvEBAAJrsQWW12uyF2MSrOt1pNjA== 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-09-16T06:47:44.738 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB01OdmuvFoDRAA7mY3fueoFHGIUcFfCsc+ng== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB31Odmh0SUEhAAw1nC2Y4Ge0LkLKuLxqxWQw== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.10] 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB61OdmKDKzGRAAsDdsxeBnxjvmn9coqi4ovg== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.14] 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB91OdmyVx5HxAASkjy6H7/fGpAlmKdBWkwrA== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBS1OdmeQd4KBAAOFsHRHov6Yp7H9ZUGaUOXA== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCD1OdmpuJ9HRAAgPfQ8685ODfgconuo9OxXw== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCG1OdmV1akIRAAD+fCd4M4DIFwKe6cYRxn4w== 2024-09-16T06:47:44.739 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.11] 2024-09-16T06:47:44.740 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCJ1OdmUPboJxAAMX8phwCREmuz9lH67Sf+DA== 2024-09-16T06:47:44.740 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.15] 2024-09-16T06:47:44.740 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCM1OdmvbjOLxAALups8ph+/4qm5aDBZLUwSg== 2024-09-16T06:47:44.740 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-16T06:47:44.740 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBS1OdmP8aDLRAATf5t1GImUvVZE9LsOEvnXw== 2024-09-16T06:47:44.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.y] 2024-09-16T06:47:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBS1OdmPJnJJRAA31BkN+jCB28u2yyIJXMl0w== 2024-09-16T06:47:44.741 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.0] 2024-09-16T06:47:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBV1Odme2kOJxAAtPNbNXqWG8f7QMvxlFlcew== 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.4] 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBY1Odm19stLBAADGiF4WSOtT243JUWTXD0Gg== 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.8] 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBb1Odmdm5+NBAAzxC4wAr88JVPPFaNa1wzFw== 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.12] 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBe1Odmd1QDOxAAZ0wwJsF5txNQ5XhhH8lx2A== 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.1] 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBk1Odmfb7cNhAAFSGmqvFMAQm+kp1tYH0ZHg== 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.5] 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBo1Odm8Dk4ARAAxH6mjclBN8As2xQZvyjzew== 2024-09-16T06:47:44.742 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.9] 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBr1OdmnermCRAA1pTYeThGPNX7FFKrJPOoCA== 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.13] 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBu1Odmq6MvEBAAJrsQWW12uyF2MSrOt1pNjA== 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.2] 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQB01OdmuvFoDRAA7mY3fueoFHGIUcFfCsc+ng== 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.6] 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQB31Odmh0SUEhAAw1nC2Y4Ge0LkLKuLxqxWQw== 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.10] 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQB61OdmKDKzGRAAsDdsxeBnxjvmn9coqi4ovg== 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.14] 2024-09-16T06:47:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQB91OdmyVx5HxAASkjy6H7/fGpAlmKdBWkwrA== 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.x] 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBS1OdmeQd4KBAAOFsHRHov6Yp7H9ZUGaUOXA== 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.3] 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCD1OdmpuJ9HRAAgPfQ8685ODfgconuo9OxXw== 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.7] 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCG1OdmV1akIRAAD+fCd4M4DIFwKe6cYRxn4w== 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.11] 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCJ1OdmUPboJxAAMX8phwCREmuz9lH67Sf+DA== 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.15] 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCM1OdmvbjOLxAALups8ph+/4qm5aDBZLUwSg== 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-09-16T06:47:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBS1OdmP8aDLRAATf5t1GImUvVZE9LsOEvnXw== 2024-09-16T06:47:44.745 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2024-09-16T06:47:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBS1OdmPJnJJRAA31BkN+jCB28u2yyIJXMl0w== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBV1Odme2kOJxAAtPNbNXqWG8f7QMvxlFlcew== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBY1Odm19stLBAADGiF4WSOtT243JUWTXD0Gg== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.8] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBb1Odmdm5+NBAAzxC4wAr88JVPPFaNa1wzFw== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.12] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBe1Odmd1QDOxAAZ0wwJsF5txNQ5XhhH8lx2A== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBk1Odmfb7cNhAAFSGmqvFMAQm+kp1tYH0ZHg== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBo1Odm8Dk4ARAAxH6mjclBN8As2xQZvyjzew== 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.9] 2024-09-16T06:47:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBr1OdmnermCRAA1pTYeThGPNX7FFKrJPOoCA== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.13] 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBu1Odmq6MvEBAAJrsQWW12uyF2MSrOt1pNjA== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQB01OdmuvFoDRAA7mY3fueoFHGIUcFfCsc+ng== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQB31Odmh0SUEhAAw1nC2Y4Ge0LkLKuLxqxWQw== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.10] 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQB61OdmKDKzGRAAsDdsxeBnxjvmn9coqi4ovg== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.14] 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQB91OdmyVx5HxAASkjy6H7/fGpAlmKdBWkwrA== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBS1OdmeQd4KBAAOFsHRHov6Yp7H9ZUGaUOXA== 2024-09-16T06:47:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCD1OdmpuJ9HRAAgPfQ8685ODfgconuo9OxXw== 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCG1OdmV1akIRAAD+fCd4M4DIFwKe6cYRxn4w== 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.11] 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCJ1OdmUPboJxAAMX8phwCREmuz9lH67Sf+DA== 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.15] 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCM1OdmvbjOLxAALups8ph+/4qm5aDBZLUwSg== 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-09-16T06:47:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBS1OdmP8aDLRAATf5t1GImUvVZE9LsOEvnXw== 2024-09-16T06:47:44.748 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T06:47:44.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T06:47:44.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T06:47:44.835 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:44.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:44.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:44.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:44.958 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:44.960 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.087 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.090 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.282 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.334 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.417 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.446 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.506 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.531 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.665 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.750 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T06:47:45.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T06:47:45.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T06:47:45.907 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.922 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:45.989 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.078 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T06:47:46.253 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T06:47:46.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T06:47:46.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T06:47:46.305 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T06:47:46.306 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T06:47:46.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T06:47:46.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T06:47:46.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T06:47:46.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T06:47:46.530 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T06:47:46.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T06:47:46.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T06:47:46.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T06:47:46.658 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:46.660 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:46.662 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T06:47:46.703 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T06:47:46.704 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T06:47:46.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T06:47:46.706 INFO:tasks.ceph.mon.a:Started 2024-09-16T06:47:46.707 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T06:47:46.707 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T06:47:46.711 INFO:tasks.ceph.mon.b:Started 2024-09-16T06:47:46.711 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T06:47:46.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T06:47:46.750 INFO:tasks.ceph.mon.c:Started 2024-09-16T06:47:46.750 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T06:47:46.750 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T06:47:46.750 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T06:47:46.753 INFO:tasks.ceph.mgr.y:Started 2024-09-16T06:47:46.753 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T06:47:46.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T06:47:46.755 INFO:tasks.ceph.mgr.x:Started 2024-09-16T06:47:46.755 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T06:47:46.756 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T06:47:53.315 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-09-16T06:47:53.334 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T06:47:53.334 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T06:47:53.335 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:53.335 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T06:47:53.345 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:53.345 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T06:47:53.398 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:53.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-16T06:47:53.419 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:53.418+0000 7faad57af180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:47:53.451 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T06:47:53.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-09-16T06:47:53.473 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:53.472+0000 7efe1c8c1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T06:47:53.484 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:53.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T06:47:53.499 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:53.499 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T06:47:53.553 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:53.553 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-16T06:47:53.609 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-16T06:47:53.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-09-16T06:47:53.666 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:53.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T06:47:53.681 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:53.681 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T06:47:53.737 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:53.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-16T06:47:53.791 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-16T06:47:53.791 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-09-16T06:47:53.844 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:53.845 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T06:47:53.858 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:53.858 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T06:47:53.911 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:53.911 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-16T06:47:53.912 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:53.910+0000 7faad57af180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:47:53.959 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:53.956+0000 7efe1c8c1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T06:47:53.967 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-16T06:47:53.968 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-09-16T06:47:54.024 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 54494294-f25f-41f2-bff7-106492665e07 0 2024-09-16T06:47:54.104 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.102+0000 7faad57af180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:47:54.167 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:54.164+0000 7efe1c8c1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T06:47:54.275 INFO:teuthology.orchestra.run.smithi117.stdout:0 2024-09-16T06:47:54.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new a265e752-1cd6-497e-bcb9-c56591a5ce30 1 2024-09-16T06:47:54.530 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-16T06:47:54.549 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new dada34c6-4dc6-4f9f-bc31-85983fd2d7fd 2 2024-09-16T06:47:54.658 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.658+0000 7faad57af180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:47:54.707 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:54.704+0000 7efe1c8c1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T06:47:54.713 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.714+0000 7faad57af180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:47:54.753 INFO:teuthology.orchestra.run.smithi117.stdout:2 2024-09-16T06:47:54.761 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:54.760+0000 7efe1c8c1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T06:47:54.767 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.766+0000 7faad57af180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:47:54.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 6c0fc051-a4a0-4ce5-989a-668e75bbe3c0 3 2024-09-16T06:47:54.813 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:54.812+0000 7efe1c8c1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T06:47:54.847 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.846+0000 7faad57af180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:47:54.889 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:54.888+0000 7efe1c8c1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T06:47:54.900 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.902+0000 7faad57af180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:47:54.941 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:54.940+0000 7efe1c8c1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T06:47:54.968 INFO:teuthology.orchestra.run.smithi117.stdout:3 2024-09-16T06:47:54.978 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:54.978+0000 7faad57af180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:47:54.984 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 36a46387-b859-4937-bbd0-5dd708630c6e 4 2024-09-16T06:47:55.017 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:55.016+0000 7efe1c8c1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T06:47:55.031 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:55.034+0000 7faad57af180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:47:55.068 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:55.064+0000 7efe1c8c1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T06:47:55.181 INFO:teuthology.orchestra.run.smithi117.stdout:4 2024-09-16T06:47:55.185 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:55.186+0000 7faad57af180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:47:55.197 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 073f3d23-77f0-4fd3-b067-27fa81137288 5 2024-09-16T06:47:55.220 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:55.220+0000 7efe1c8c1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T06:47:55.247 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:55.246+0000 7faad57af180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:47:55.279 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:55.276+0000 7efe1c8c1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T06:47:55.407 INFO:teuthology.orchestra.run.smithi117.stdout:5 2024-09-16T06:47:55.423 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 8a81592f-3704-48f6-89ba-cc4997c2105e 6 2024-09-16T06:47:55.669 INFO:teuthology.orchestra.run.smithi117.stdout:6 2024-09-16T06:47:55.685 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27 7 2024-09-16T06:47:55.876 INFO:tasks.ceph.mgr.x.smithi117.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T06:47:55.876 INFO:tasks.ceph.mgr.x.smithi117.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T06:47:55.876 INFO:tasks.ceph.mgr.x.smithi117.stderr: from numpy import show_config as show_numpy_config 2024-09-16T06:47:55.889 INFO:teuthology.orchestra.run.smithi117.stdout:7 2024-09-16T06:47:55.907 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 7ce38131-057f-4d45-88a5-dae8bcceae3c 8 2024-09-16T06:47:55.920 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:55.922+0000 7faad57af180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:47:55.977 INFO:tasks.ceph.mgr.y.smithi100.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T06:47:55.977 INFO:tasks.ceph.mgr.y.smithi100.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T06:47:55.977 INFO:tasks.ceph.mgr.y.smithi100.stderr: from numpy import show_config as show_numpy_config 2024-09-16T06:47:56.049 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:56.048+0000 7efe1c8c1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T06:47:56.054 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:56.054+0000 7faad57af180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:47:56.110 INFO:teuthology.orchestra.run.smithi117.stdout:8 2024-09-16T06:47:56.127 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new aa7fde6e-6563-4e37-8af2-9114d0091362 9 2024-09-16T06:47:56.214 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:56.214+0000 7faad57af180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:47:56.232 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:56.228+0000 7efe1c8c1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:47:56.279 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:56.278+0000 7faad57af180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:47:56.369 INFO:teuthology.orchestra.run.smithi117.stdout:9 2024-09-16T06:47:56.385 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 05c5389d-8082-4be3-aa6d-138e00720f06 10 2024-09-16T06:47:56.399 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:56.396+0000 7efe1c8c1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T06:47:56.416 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:56.418+0000 7faad57af180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:47:56.469 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:56.468+0000 7efe1c8c1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T06:47:56.589 INFO:teuthology.orchestra.run.smithi117.stdout:10 2024-09-16T06:47:56.604 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 99552590-1d0a-420e-b0c3-357246880658 11 2024-09-16T06:47:56.616 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:56.616+0000 7efe1c8c1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T06:47:56.849 INFO:teuthology.orchestra.run.smithi117.stdout:11 2024-09-16T06:47:56.865 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 165fea1c-74b6-4806-a0d9-47441e161ed5 12 2024-09-16T06:47:57.027 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:57.026+0000 7faad57af180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:47:57.076 INFO:teuthology.orchestra.run.smithi117.stdout:12 2024-09-16T06:47:57.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 03402d0c-e7fd-4233-b2cd-bf79cd75e9bc 13 2024-09-16T06:47:57.104 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:57.106+0000 7faad57af180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:47:57.264 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:57.264+0000 7efe1c8c1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T06:47:57.287 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:57.286+0000 7faad57af180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:47:57.297 INFO:teuthology.orchestra.run.smithi117.stdout:13 2024-09-16T06:47:57.313 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 7b027faf-5c9e-4aee-9cb9-9c6459b8fe39 14 2024-09-16T06:47:57.340 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:57.336+0000 7efe1c8c1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T06:47:57.363 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:57.362+0000 7faad57af180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:47:57.518 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:57.516+0000 7efe1c8c1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T06:47:57.590 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:57.588+0000 7efe1c8c1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T06:47:57.652 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:57.654+0000 7faad57af180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:47:57.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-16T06:47:57.686+0000 7ff8a9a9e640 0 --2- 172.21.15.117:0/1972549971 >> v2:172.21.15.100:3300/0 conn(0x7ff8a411adc0 0x7ff8a411d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T06:47:57.714 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:57.714+0000 7faad57af180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:47:57.787 INFO:teuthology.orchestra.run.smithi117.stdout:14 2024-09-16T06:47:57.806 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 4877aa40-c0d8-494f-8a98-36089ed694c2 15 2024-09-16T06:47:57.886 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:57.884+0000 7efe1c8c1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T06:47:57.946 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-16T06:47:57.944+0000 7efe1c8c1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T06:47:58.046 INFO:teuthology.orchestra.run.smithi117.stdout:15 2024-09-16T06:47:58.063 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T06:47:58.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T06:47:58.066 INFO:tasks.ceph.osd.0:Started 2024-09-16T06:47:58.066 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T06:47:58.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T06:47:58.068 INFO:tasks.ceph.osd.4:Started 2024-09-16T06:47:58.068 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-16T06:47:58.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-16T06:47:58.071 INFO:tasks.ceph.osd.8:Started 2024-09-16T06:47:58.071 INFO:tasks.ceph.osd.12:Restarting daemon 2024-09-16T06:47:58.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-09-16T06:47:58.073 INFO:tasks.ceph.osd.12:Started 2024-09-16T06:47:58.073 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T06:47:58.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T06:47:58.076 INFO:tasks.ceph.osd.1:Started 2024-09-16T06:47:58.076 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T06:47:58.076 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T06:47:58.079 INFO:tasks.ceph.osd.5:Started 2024-09-16T06:47:58.079 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-16T06:47:58.079 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-16T06:47:58.081 INFO:tasks.ceph.osd.9:Started 2024-09-16T06:47:58.081 INFO:tasks.ceph.osd.13:Restarting daemon 2024-09-16T06:47:58.083 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-09-16T06:47:58.085 INFO:tasks.ceph.osd.13:Started 2024-09-16T06:47:58.085 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T06:47:58.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T06:47:58.088 INFO:tasks.ceph.osd.2:Started 2024-09-16T06:47:58.088 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T06:47:58.088 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T06:47:58.090 INFO:tasks.ceph.osd.6:Started 2024-09-16T06:47:58.091 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-16T06:47:58.091 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-16T06:47:58.093 INFO:tasks.ceph.osd.10:Started 2024-09-16T06:47:58.093 INFO:tasks.ceph.osd.14:Restarting daemon 2024-09-16T06:47:58.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-09-16T06:47:58.096 INFO:tasks.ceph.osd.14:Started 2024-09-16T06:47:58.096 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T06:47:58.096 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T06:47:58.098 INFO:tasks.ceph.osd.3:Started 2024-09-16T06:47:58.098 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T06:47:58.098 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T06:47:58.100 INFO:tasks.ceph.osd.7:Started 2024-09-16T06:47:58.100 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-16T06:47:58.100 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-16T06:47:58.102 INFO:tasks.ceph.osd.11:Started 2024-09-16T06:47:58.102 INFO:tasks.ceph.osd.15:Restarting daemon 2024-09-16T06:47:58.103 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-09-16T06:47:58.104 INFO:tasks.ceph.osd.15:Started 2024-09-16T06:47:58.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:47:58.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:47:58.285 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":18,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:47:58.043572+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T06:47:58.043572+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:47:58.299 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T06:47:58.299 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T06:47:58.670 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:47:58.668+0000 7f5c21fa37c0 -1 Falling back to public interface 2024-09-16T06:47:58.673 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:47:58.672+0000 7fd86cdcd7c0 -1 Falling back to public interface 2024-09-16T06:47:58.693 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:47:58.692+0000 7f9a049777c0 -1 Falling back to public interface 2024-09-16T06:47:58.705 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:47:58.704+0000 7f89645a67c0 -1 Falling back to public interface 2024-09-16T06:47:58.707 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:47:58.710+0000 7f677df2c7c0 -1 Falling back to public interface 2024-09-16T06:47:58.709 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:47:58.708+0000 7f4d6503c7c0 -1 Falling back to public interface 2024-09-16T06:47:58.709 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:47:58.708+0000 7fc39adad7c0 -1 Falling back to public interface 2024-09-16T06:47:58.713 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:47:58.712+0000 7fd4e494d7c0 -1 Falling back to public interface 2024-09-16T06:47:58.713 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:47:58.714+0000 7fa1198917c0 -1 Falling back to public interface 2024-09-16T06:47:58.717 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:47:58.716+0000 7f1071c4a7c0 -1 Falling back to public interface 2024-09-16T06:47:58.718 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:47:58.722+0000 7fd5bee2e7c0 -1 Falling back to public interface 2024-09-16T06:47:58.718 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:47:58.722+0000 7f6a7c3aa7c0 -1 Falling back to public interface 2024-09-16T06:47:58.726 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:47:58.730+0000 7fd09cac47c0 -1 Falling back to public interface 2024-09-16T06:47:58.736 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:47:58.738+0000 7fb886c197c0 -1 Falling back to public interface 2024-09-16T06:47:58.753 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:58.754+0000 7faa7cd62640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T06:47:58.754 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:47:58.754+0000 7faa7cd62640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T06:47:58.756 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:47:58.758+0000 7f71e48f57c0 -1 Falling back to public interface 2024-09-16T06:47:58.760 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:47:58.762+0000 7f397c44e7c0 -1 Falling back to public interface 2024-09-16T06:47:59.912 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T06:48:00.208 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-16T06:48:00.208 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":18,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:47:58.043572+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T06:47:58.043572+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:48:00.225 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-09-16T06:48:02.403 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:02.406+0000 7f677df2c7c0 -1 osd.13 0 log_to_monitors true 2024-09-16T06:48:02.411 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:02.410+0000 7fa1198917c0 -1 osd.3 0 log_to_monitors true 2024-09-16T06:48:02.467 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:02.464+0000 7f4d6503c7c0 -1 osd.10 0 log_to_monitors true 2024-09-16T06:48:02.490 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:02.488+0000 7f5c21fa37c0 -1 osd.4 0 log_to_monitors true 2024-09-16T06:48:02.499 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:02.498+0000 7fb886c197c0 -1 osd.11 0 log_to_monitors true 2024-09-16T06:48:02.503 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:02.506+0000 7f6a7c3aa7c0 -1 osd.5 0 log_to_monitors true 2024-09-16T06:48:02.562 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:02.566+0000 7fd09cac47c0 -1 osd.9 0 log_to_monitors true 2024-09-16T06:48:02.569 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:02.568+0000 7f1071c4a7c0 -1 osd.6 0 log_to_monitors true 2024-09-16T06:48:02.570 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:02.568+0000 7f89645a67c0 -1 osd.2 0 log_to_monitors true 2024-09-16T06:48:02.579 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:02.578+0000 7f397c44e7c0 -1 osd.15 0 log_to_monitors true 2024-09-16T06:48:02.592 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:02.592+0000 7fd86cdcd7c0 -1 osd.0 0 log_to_monitors true 2024-09-16T06:48:02.603 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:02.606+0000 7f71e48f57c0 -1 osd.7 0 log_to_monitors true 2024-09-16T06:48:02.609 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:02.608+0000 7f9a049777c0 -1 osd.8 0 log_to_monitors true 2024-09-16T06:48:02.614 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:02.612+0000 7fc39adad7c0 -1 osd.12 0 log_to_monitors true 2024-09-16T06:48:02.627 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:02.630+0000 7fd5bee2e7c0 -1 osd.1 0 log_to_monitors true 2024-09-16T06:48:02.628 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:02.628+0000 7fd4e494d7c0 -1 osd.14 0 log_to_monitors true 2024-09-16T06:48:03.495 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:03.494+0000 7fa115013640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T06:48:03.513 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:03.514+0000 7fa111660640 -1 osd.3 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.342 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:04.346+0000 7f677aee9640 -1 osd.13 0 waiting for initial osdmap 2024-09-16T06:48:04.395 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:04.398+0000 7f6771cf3640 -1 osd.13 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.503 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:04.500+0000 7f5c1d725640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T06:48:04.503 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:04.500+0000 7fc39652f640 -1 osd.12 0 waiting for initial osdmap 2024-09-16T06:48:04.512 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:04.512+0000 7f106d3cc640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T06:48:04.512 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:04.514+0000 7f71e0077640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T06:48:04.518 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:04.522+0000 7f6a77b2c640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T06:48:04.518 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:04.516+0000 7f9a000f9640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T06:48:04.523 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:04.520+0000 7f5c19d72640 -1 osd.4 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.528 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:04.530+0000 7f6a74179640 -1 osd.5 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.534 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:04.534+0000 7f71dc6c4640 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.547 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:04.544+0000 7f99fc746640 -1 osd.8 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.548 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:04.544+0000 7fc392b7c640 -1 osd.12 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:04.730 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:04.728+0000 7f1069a19640 -1 osd.6 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:05.417 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:05.416+0000 7fd86854f640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T06:48:05.440 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:05.440+0000 7fd860b94640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:05.843 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:05.842+0000 7f397940b640 -1 osd.15 0 waiting for initial osdmap 2024-09-16T06:48:05.854 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:05.854+0000 7fb88239b640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T06:48:05.868 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:05.870+0000 7f3970215640 -1 osd.15 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:05.876 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:05.878+0000 7fb87a9e0640 -1 osd.11 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:05.936 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:05.938+0000 7fd098246640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T06:48:05.962 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:05.966+0000 7fd09088b640 -1 osd.9 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:05.988 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:05.990+0000 7fd5ba5b0640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T06:48:06.018 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:06.022+0000 7fd5b2bf5640 -1 osd.1 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:06.301 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:06.300+0000 7f8961563640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T06:48:06.325 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:06.324+0000 7f895836d640 -1 osd.2 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:06.347 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:06.344+0000 7f4d607be640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T06:48:06.371 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:06.368+0000 7fd4e00cf640 -1 osd.14 0 waiting for initial osdmap 2024-09-16T06:48:06.371 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:06.368+0000 7f4d58e03640 -1 osd.10 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:06.401 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:06.400+0000 7fd4d8714640 -1 osd.14 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:48:07.834 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T06:48:08.142 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-16T06:48:08.143 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":24,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:48:07.625282+0000","last_up_change":"2024-09-16T06:48:06.616578+0000","last_in_change":"2024-09-16T06:47:58.043572+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6812","nonce":1081990569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6816","nonce":1081990569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6823","nonce":1081990569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6820","nonce":1081990569}]},"public_addr":"172.21.15.107:6812/1081990569","cluster_addr":"172.21.15.107:6816/1081990569","heartbeat_back_addr":"172.21.15.107:6823/1081990569","heartbeat_front_addr":"172.21.15.107:6820/1081990569","state":["exists","up"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.584963+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.677023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.608639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.384489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.480551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.476690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.615096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.581224+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.597357+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.549502+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.448485+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.495825+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.628407+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.368452+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.600045+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.567594+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:48:08.160 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-09-16T06:48:08.160 INFO:tasks.ceph:Creating RBD pool 2024-09-16T06:48:08.161 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T06:48:08.648 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'rbd' created 2024-09-16T06:48:08.666 DEBUG:teuthology.orchestra.run.smithi100:> rbd --cluster ceph pool init rbd 2024-09-16T06:48:11.791 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T06:48:11.791 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T06:48:11.791 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T06:48:11.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T06:48:11.792 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T06:48:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:12.161 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3382893237},{"type":"v1","addr":"172.21.15.117:6801","nonce":3382893237}]},"active_addr":"172.21.15.117:6801/3382893237","active_change":"2024-09-16T06:47:57.737243+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":457025735}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":899211195}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":132275295}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1243534586}]}]} 2024-09-16T06:48:12.165 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T06:48:12.165 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T06:48:12.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:48:12.472 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:12.472 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:48:11.754245+0000","last_up_change":"2024-09-16T06:48:06.616578+0000","last_in_change":"2024-09-16T06:47:58.043572+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T06:48:08.489111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6812","nonce":1081990569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6816","nonce":1081990569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6823","nonce":1081990569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6820","nonce":1081990569}]},"public_addr":"172.21.15.107:6812/1081990569","cluster_addr":"172.21.15.107:6816/1081990569","heartbeat_back_addr":"172.21.15.107:6823/1081990569","heartbeat_front_addr":"172.21.15.107:6820/1081990569","state":["exists","up"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.584963+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.677023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.608639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.384489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.480551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.476690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.615096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.581224+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.597357+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.549502+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.448485+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.495825+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.628407+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.368452+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.600045+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.567594+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:48:12.491 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T06:48:12.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:48:12.791 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:12.792 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":28,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:48:11.754245+0000","last_up_change":"2024-09-16T06:48:06.616578+0000","last_in_change":"2024-09-16T06:47:58.043572+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T06:48:08.489111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6812","nonce":1081990569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6816","nonce":1081990569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6823","nonce":1081990569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6820","nonce":1081990569}]},"public_addr":"172.21.15.107:6812/1081990569","cluster_addr":"172.21.15.107:6816/1081990569","heartbeat_back_addr":"172.21.15.107:6823/1081990569","heartbeat_front_addr":"172.21.15.107:6820/1081990569","state":["exists","up"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.584963+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.677023+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.608639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.384489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.480551+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.476690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.615096+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.581224+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.597357+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.549502+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.448485+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.495825+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.628407+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.368452+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.600045+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.567594+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:48:12.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T06:48:12.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T06:48:12.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T06:48:12.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T06:48:12.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T06:48:12.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T06:48:12.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T06:48:12.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T06:48:12.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-16T06:48:12.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-16T06:48:12.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-16T06:48:12.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-16T06:48:12.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-16T06:48:12.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-16T06:48:12.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-16T06:48:12.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-16T06:48:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:90194313220 2024-09-16T06:48:13.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T06:48:13.257 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:13.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T06:48:13.279 INFO:teuthology.orchestra.run.smithi100.stdout:98784247812 2024-09-16T06:48:13.280 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-16T06:48:13.294 INFO:teuthology.orchestra.run.smithi100.stdout:94489280517 2024-09-16T06:48:13.294 INFO:teuthology.orchestra.run.smithi100.stdout:90194313220 2024-09-16T06:48:13.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-16T06:48:13.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-16T06:48:13.316 INFO:teuthology.orchestra.run.smithi100.stdout:98784247811 2024-09-16T06:48:13.316 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-16T06:48:13.317 INFO:teuthology.orchestra.run.smithi100.stdout:98784247813 2024-09-16T06:48:13.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-16T06:48:13.341 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:13.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T06:48:13.346 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:13.348 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T06:48:13.349 INFO:teuthology.orchestra.run.smithi100.stdout:98784247812 2024-09-16T06:48:13.349 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T06:48:13.350 INFO:teuthology.orchestra.run.smithi100.stdout:98784247814 2024-09-16T06:48:13.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T06:48:13.351 INFO:teuthology.orchestra.run.smithi100.stdout:98784247811 2024-09-16T06:48:13.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-16T06:48:13.362 INFO:teuthology.orchestra.run.smithi100.stdout:98784247813 2024-09-16T06:48:13.362 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T06:48:13.384 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:13.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-16T06:48:13.388 INFO:teuthology.orchestra.run.smithi100.stdout:98784247812 2024-09-16T06:48:13.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-16T06:48:13.473 INFO:teuthology.orchestra.run.smithi100.stdout:94489280517 2024-09-16T06:48:13.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T06:48:14.114 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:14.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.4 2024-09-16T06:48:14.176 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.211 INFO:teuthology.orchestra.run.smithi100.stdout:98784247814 2024-09-16T06:48:14.239 INFO:teuthology.orchestra.run.smithi100.stdout:98784247813 2024-09-16T06:48:14.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247814 got 98784247814 for osd.1 2024-09-16T06:48:14.253 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.274 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:14.281 INFO:teuthology.orchestra.run.smithi100.stdout:98784247811 2024-09-16T06:48:14.284 INFO:teuthology.orchestra.run.smithi100.stdout:98784247813 2024-09-16T06:48:14.291 INFO:teuthology.orchestra.run.smithi100.stdout:90194313220 2024-09-16T06:48:14.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.0 2024-09-16T06:48:14.292 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.5 2024-09-16T06:48:14.316 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.14 2024-09-16T06:48:14.332 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.336 INFO:teuthology.orchestra.run.smithi100.stdout:94489280517 2024-09-16T06:48:14.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.3 2024-09-16T06:48:14.343 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.9 2024-09-16T06:48:14.344 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.373 INFO:teuthology.orchestra.run.smithi100.stdout:90194313220 2024-09-16T06:48:14.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.12 2024-09-16T06:48:14.378 INFO:teuthology.orchestra.run.smithi100.stdout:98784247812 2024-09-16T06:48:14.378 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:14.378 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.389 INFO:teuthology.orchestra.run.smithi100.stdout:98784247812 2024-09-16T06:48:14.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.13 2024-09-16T06:48:14.405 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.6 2024-09-16T06:48:14.407 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.10 2024-09-16T06:48:14.418 INFO:teuthology.orchestra.run.smithi100.stdout:98784247811 2024-09-16T06:48:14.419 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.430 INFO:teuthology.orchestra.run.smithi100.stdout:94489280516 2024-09-16T06:48:14.431 INFO:teuthology.orchestra.run.smithi100.stdout:98784247812 2024-09-16T06:48:14.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.2 2024-09-16T06:48:14.443 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.15 2024-09-16T06:48:14.444 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.8 2024-09-16T06:48:14.454 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.11 2024-09-16T06:48:14.454 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.458 INFO:teuthology.orchestra.run.smithi100.stdout:94489280517 2024-09-16T06:48:14.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.7 2024-09-16T06:48:14.475 DEBUG:teuthology.parallel:result is None 2024-09-16T06:48:14.475 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T06:48:14.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:48:14.795 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:14.796 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:48:14.811 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-16T06:48:13.746798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":438416,"kb_used_data":5624,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511024,"statfs":{"total":1546188226560,"available":1545739288576,"internally_reserved":0,"allocated":5758976,"data_stored":2924434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":442473568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":16,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000610"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:13.218482+0000","last_change":"2024-09-16T06:48:13.218911+0000","last_active":"2024-09-16T06:48:13.218482+0000","last_peered":"2024-09-16T06:48:13.218482+0000","last_clean":"2024-09-16T06:48:13.218482+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:48:13.218482+0000","last_undegraded":"2024-09-16T06:48:13.218482+0000","last_fullsized":"2024-09-16T06:48:13.218482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:09.738021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.775015+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:48:11.775015+0000","last_peered":"2024-09-16T06:48:11.775015+0000","last_clean":"2024-09-16T06:48:11.775015+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:48:11.775015+0000","last_undegraded":"2024-09-16T06:48:11.775015+0000","last_fullsized":"2024-09-16T06:48:11.775015+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.416727+0000","last_change":"2024-09-16T06:48:12.417012+0000","last_active":"2024-09-16T06:48:12.416727+0000","last_peered":"2024-09-16T06:48:12.416727+0000","last_clean":"2024-09-16T06:48:12.416727+0000","last_became_active":"2024-09-16T06:48:09.894853+0000","last_became_peered":"2024-09-16T06:48:09.894853+0000","last_unstale":"2024-09-16T06:48:12.416727+0000","last_undegraded":"2024-09-16T06:48:12.416727+0000","last_fullsized":"2024-09-16T06:48:12.416727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:26.719521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.417890+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:48:12.417890+0000","last_peered":"2024-09-16T06:48:12.417890+0000","last_clean":"2024-09-16T06:48:12.417890+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:48:12.417890+0000","last_undegraded":"2024-09-16T06:48:12.417890+0000","last_fullsized":"2024-09-16T06:48:12.417890+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.768932+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:48:11.768932+0000","last_peered":"2024-09-16T06:48:11.768932+0000","last_clean":"2024-09-16T06:48:11.768932+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:48:11.768932+0000","last_undegraded":"2024-09-16T06:48:11.768932+0000","last_fullsized":"2024-09-16T06:48:11.768932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636861+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:48:12.636861+0000","last_peered":"2024-09-16T06:48:12.636861+0000","last_clean":"2024-09-16T06:48:12.636861+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:48:12.636861+0000","last_undegraded":"2024-09-16T06:48:12.636861+0000","last_fullsized":"2024-09-16T06:48:12.636861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.766789+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:48:11.766789+0000","last_peered":"2024-09-16T06:48:11.766789+0000","last_clean":"2024-09-16T06:48:11.766789+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:48:11.766789+0000","last_undegraded":"2024-09-16T06:48:11.766789+0000","last_fullsized":"2024-09-16T06:48:11.766789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:16.422076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636954+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:48:12.636954+0000","last_peered":"2024-09-16T06:48:12.636954+0000","last_clean":"2024-09-16T06:48:12.636954+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:48:12.636954+0000","last_undegraded":"2024-09-16T06:48:12.636954+0000","last_fullsized":"2024-09-16T06:48:12.636954+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"28'157","reported_seq":209,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.027613+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:48:12.027613+0000","last_peered":"2024-09-16T06:48:12.027613+0000","last_clean":"2024-09-16T06:48:12.027613+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:48:12.027613+0000","last_undegraded":"2024-09-16T06:48:12.027613+0000","last_fullsized":"2024-09-16T06:48:12.027613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.138478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247811,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":288.66899999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6559999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]}]},{"osd":14,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9359999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]}]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.673}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]}]},{"osd":3,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.984}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]}]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":4,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,5,6,7,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]}]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9390000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2679999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":8,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3330000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":2,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1909999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]},{"osd":9,"up_from":23,"seq":98784247811,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":104.761}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0190000000000001}]}]},{"osd":10,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,5,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.089}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:48:14.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:48:15.100 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:15.101 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:48:15.117 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-16T06:48:13.746798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":438416,"kb_used_data":5624,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511024,"statfs":{"total":1546188226560,"available":1545739288576,"internally_reserved":0,"allocated":5758976,"data_stored":2924434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":442473568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":16,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000610"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:13.218482+0000","last_change":"2024-09-16T06:48:13.218911+0000","last_active":"2024-09-16T06:48:13.218482+0000","last_peered":"2024-09-16T06:48:13.218482+0000","last_clean":"2024-09-16T06:48:13.218482+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:48:13.218482+0000","last_undegraded":"2024-09-16T06:48:13.218482+0000","last_fullsized":"2024-09-16T06:48:13.218482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:09.738021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.775015+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:48:11.775015+0000","last_peered":"2024-09-16T06:48:11.775015+0000","last_clean":"2024-09-16T06:48:11.775015+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:48:11.775015+0000","last_undegraded":"2024-09-16T06:48:11.775015+0000","last_fullsized":"2024-09-16T06:48:11.775015+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.416727+0000","last_change":"2024-09-16T06:48:12.417012+0000","last_active":"2024-09-16T06:48:12.416727+0000","last_peered":"2024-09-16T06:48:12.416727+0000","last_clean":"2024-09-16T06:48:12.416727+0000","last_became_active":"2024-09-16T06:48:09.894853+0000","last_became_peered":"2024-09-16T06:48:09.894853+0000","last_unstale":"2024-09-16T06:48:12.416727+0000","last_undegraded":"2024-09-16T06:48:12.416727+0000","last_fullsized":"2024-09-16T06:48:12.416727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:26.719521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.417890+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:48:12.417890+0000","last_peered":"2024-09-16T06:48:12.417890+0000","last_clean":"2024-09-16T06:48:12.417890+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:48:12.417890+0000","last_undegraded":"2024-09-16T06:48:12.417890+0000","last_fullsized":"2024-09-16T06:48:12.417890+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.768932+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:48:11.768932+0000","last_peered":"2024-09-16T06:48:11.768932+0000","last_clean":"2024-09-16T06:48:11.768932+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:48:11.768932+0000","last_undegraded":"2024-09-16T06:48:11.768932+0000","last_fullsized":"2024-09-16T06:48:11.768932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636861+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:48:12.636861+0000","last_peered":"2024-09-16T06:48:12.636861+0000","last_clean":"2024-09-16T06:48:12.636861+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:48:12.636861+0000","last_undegraded":"2024-09-16T06:48:12.636861+0000","last_fullsized":"2024-09-16T06:48:12.636861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.766789+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:48:11.766789+0000","last_peered":"2024-09-16T06:48:11.766789+0000","last_clean":"2024-09-16T06:48:11.766789+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:48:11.766789+0000","last_undegraded":"2024-09-16T06:48:11.766789+0000","last_fullsized":"2024-09-16T06:48:11.766789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:16.422076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636954+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:48:12.636954+0000","last_peered":"2024-09-16T06:48:12.636954+0000","last_clean":"2024-09-16T06:48:12.636954+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:48:12.636954+0000","last_undegraded":"2024-09-16T06:48:12.636954+0000","last_fullsized":"2024-09-16T06:48:12.636954+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"28'157","reported_seq":209,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.027613+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:48:12.027613+0000","last_peered":"2024-09-16T06:48:12.027613+0000","last_clean":"2024-09-16T06:48:12.027613+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:48:12.027613+0000","last_undegraded":"2024-09-16T06:48:12.027613+0000","last_fullsized":"2024-09-16T06:48:12.027613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.138478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247811,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":288.66899999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6559999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]}]},{"osd":14,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9359999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]}]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.673}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]}]},{"osd":3,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.984}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]}]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":4,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,5,6,7,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]}]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9390000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2679999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":8,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3330000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":2,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1909999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]},{"osd":9,"up_from":23,"seq":98784247811,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":104.761}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0190000000000001}]}]},{"osd":10,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,5,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.089}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:48:15.119 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T06:48:15.119 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T06:48:15.119 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T06:48:15.119 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T06:48:15.421 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:15.421 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T06:48:15.439 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T06:48:15.439 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T06:48:15.451 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T06:48:15.451 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}} 2024-09-16T06:48:15.452 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T06:48:15.452 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T06:48:15.760 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T06:48:15.760 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T06:48:15.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:48:16.046 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:16.046 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:48:16.062 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-16T06:48:15.747601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":438416,"kb_used_data":5624,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511024,"statfs":{"total":1546188226560,"available":1545739288576,"internally_reserved":0,"allocated":5758976,"data_stored":2924434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":442473568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":16,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001412"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:13.218482+0000","last_change":"2024-09-16T06:48:13.218911+0000","last_active":"2024-09-16T06:48:13.218482+0000","last_peered":"2024-09-16T06:48:13.218482+0000","last_clean":"2024-09-16T06:48:13.218482+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:48:13.218482+0000","last_undegraded":"2024-09-16T06:48:13.218482+0000","last_fullsized":"2024-09-16T06:48:13.218482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:09.738021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.775015+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:48:11.775015+0000","last_peered":"2024-09-16T06:48:11.775015+0000","last_clean":"2024-09-16T06:48:11.775015+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:48:11.775015+0000","last_undegraded":"2024-09-16T06:48:11.775015+0000","last_fullsized":"2024-09-16T06:48:11.775015+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.416727+0000","last_change":"2024-09-16T06:48:12.417012+0000","last_active":"2024-09-16T06:48:12.416727+0000","last_peered":"2024-09-16T06:48:12.416727+0000","last_clean":"2024-09-16T06:48:12.416727+0000","last_became_active":"2024-09-16T06:48:09.894853+0000","last_became_peered":"2024-09-16T06:48:09.894853+0000","last_unstale":"2024-09-16T06:48:12.416727+0000","last_undegraded":"2024-09-16T06:48:12.416727+0000","last_fullsized":"2024-09-16T06:48:12.416727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:26.719521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.417890+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:48:12.417890+0000","last_peered":"2024-09-16T06:48:12.417890+0000","last_clean":"2024-09-16T06:48:12.417890+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:48:12.417890+0000","last_undegraded":"2024-09-16T06:48:12.417890+0000","last_fullsized":"2024-09-16T06:48:12.417890+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.768932+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:48:11.768932+0000","last_peered":"2024-09-16T06:48:11.768932+0000","last_clean":"2024-09-16T06:48:11.768932+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:48:11.768932+0000","last_undegraded":"2024-09-16T06:48:11.768932+0000","last_fullsized":"2024-09-16T06:48:11.768932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636861+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:48:12.636861+0000","last_peered":"2024-09-16T06:48:12.636861+0000","last_clean":"2024-09-16T06:48:12.636861+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:48:12.636861+0000","last_undegraded":"2024-09-16T06:48:12.636861+0000","last_fullsized":"2024-09-16T06:48:12.636861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.766789+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:48:11.766789+0000","last_peered":"2024-09-16T06:48:11.766789+0000","last_clean":"2024-09-16T06:48:11.766789+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:48:11.766789+0000","last_undegraded":"2024-09-16T06:48:11.766789+0000","last_fullsized":"2024-09-16T06:48:11.766789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:16.422076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636954+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:48:12.636954+0000","last_peered":"2024-09-16T06:48:12.636954+0000","last_clean":"2024-09-16T06:48:12.636954+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:48:12.636954+0000","last_undegraded":"2024-09-16T06:48:12.636954+0000","last_fullsized":"2024-09-16T06:48:12.636954+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"28'157","reported_seq":209,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.027613+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:48:12.027613+0000","last_peered":"2024-09-16T06:48:12.027613+0000","last_clean":"2024-09-16T06:48:12.027613+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:48:12.027613+0000","last_undegraded":"2024-09-16T06:48:12.027613+0000","last_fullsized":"2024-09-16T06:48:12.027613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.138478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247811,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":288.66899999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6559999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]}]},{"osd":14,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9359999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]}]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.673}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.984}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]}]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":4,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,5,6,7,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]}]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9390000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2679999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":8,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3330000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":2,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1909999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]},{"osd":9,"up_from":23,"seq":98784247811,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":104.761}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0190000000000001}]}]},{"osd":10,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,5,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.089}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:48:16.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:48:16.351 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:16.352 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:48:16.368 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-16T06:48:15.747601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":438416,"kb_used_data":5624,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511024,"statfs":{"total":1546188226560,"available":1545739288576,"internally_reserved":0,"allocated":5758976,"data_stored":2924434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":442473568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":16,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001412"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:13.218482+0000","last_change":"2024-09-16T06:48:13.218911+0000","last_active":"2024-09-16T06:48:13.218482+0000","last_peered":"2024-09-16T06:48:13.218482+0000","last_clean":"2024-09-16T06:48:13.218482+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:48:13.218482+0000","last_undegraded":"2024-09-16T06:48:13.218482+0000","last_fullsized":"2024-09-16T06:48:13.218482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:09.738021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.775015+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:48:11.775015+0000","last_peered":"2024-09-16T06:48:11.775015+0000","last_clean":"2024-09-16T06:48:11.775015+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:48:11.775015+0000","last_undegraded":"2024-09-16T06:48:11.775015+0000","last_fullsized":"2024-09-16T06:48:11.775015+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.416727+0000","last_change":"2024-09-16T06:48:12.417012+0000","last_active":"2024-09-16T06:48:12.416727+0000","last_peered":"2024-09-16T06:48:12.416727+0000","last_clean":"2024-09-16T06:48:12.416727+0000","last_became_active":"2024-09-16T06:48:09.894853+0000","last_became_peered":"2024-09-16T06:48:09.894853+0000","last_unstale":"2024-09-16T06:48:12.416727+0000","last_undegraded":"2024-09-16T06:48:12.416727+0000","last_fullsized":"2024-09-16T06:48:12.416727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:26.719521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.417890+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:48:12.417890+0000","last_peered":"2024-09-16T06:48:12.417890+0000","last_clean":"2024-09-16T06:48:12.417890+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:48:12.417890+0000","last_undegraded":"2024-09-16T06:48:12.417890+0000","last_fullsized":"2024-09-16T06:48:12.417890+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.768932+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:48:11.768932+0000","last_peered":"2024-09-16T06:48:11.768932+0000","last_clean":"2024-09-16T06:48:11.768932+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:48:11.768932+0000","last_undegraded":"2024-09-16T06:48:11.768932+0000","last_fullsized":"2024-09-16T06:48:11.768932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636861+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:48:12.636861+0000","last_peered":"2024-09-16T06:48:12.636861+0000","last_clean":"2024-09-16T06:48:12.636861+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:48:12.636861+0000","last_undegraded":"2024-09-16T06:48:12.636861+0000","last_fullsized":"2024-09-16T06:48:12.636861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.766789+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:48:11.766789+0000","last_peered":"2024-09-16T06:48:11.766789+0000","last_clean":"2024-09-16T06:48:11.766789+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:48:11.766789+0000","last_undegraded":"2024-09-16T06:48:11.766789+0000","last_fullsized":"2024-09-16T06:48:11.766789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:16.422076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.636954+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:48:12.636954+0000","last_peered":"2024-09-16T06:48:12.636954+0000","last_clean":"2024-09-16T06:48:12.636954+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:48:12.636954+0000","last_undegraded":"2024-09-16T06:48:12.636954+0000","last_fullsized":"2024-09-16T06:48:12.636954+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"28'157","reported_seq":209,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.027613+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:48:12.027613+0000","last_peered":"2024-09-16T06:48:12.027613+0000","last_clean":"2024-09-16T06:48:12.027613+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:48:12.027613+0000","last_undegraded":"2024-09-16T06:48:12.027613+0000","last_fullsized":"2024-09-16T06:48:12.027613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.138478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247811,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":288.66899999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6559999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]}]},{"osd":14,"up_from":23,"seq":98784247813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9359999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]}]},{"osd":11,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.673}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.984}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]}]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":4,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,5,6,7,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]}]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9390000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1850000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2679999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2149999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":8,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3330000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":2,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8560000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1909999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]},{"osd":9,"up_from":23,"seq":98784247811,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":104.761}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0190000000000001}]}]},{"osd":10,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,5,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.089}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:48:16.369 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T06:48:16.370 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T06:48:16.642 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 28 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:fsid b2976256-637c-4e28-94f6-99f20017bc5e 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-09-16T06:47:53.157733+0000 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-09-16T06:48:11.754245+0000 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:crush_version 5 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:full_ratio 0.95 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:backfillfull_ratio 0.9 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:nearfull_ratio 0.85 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:require_min_compat_client luminous 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:min_compat_client jewel 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:require_osd_release reef 2024-09-16T06:48:16.643 INFO:teuthology.orchestra.run.smithi100.stdout:stretch_mode_enabled false 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 25 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.38 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 28 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.02 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout: removed_snaps_queue [2~1] 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:max_osd 16 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/1028275949,v1:172.21.15.100:6809/1028275949] [v2:172.21.15.100:6810/1028275949,v1:172.21.15.100:6811/1028275949] exists,up 54494294-f25f-41f2-bff7-106492665e07 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/930331367,v1:172.21.15.107:6809/930331367] [v2:172.21.15.107:6811/930331367,v1:172.21.15.107:6813/930331367] exists,up a265e752-1cd6-497e-bcb9-c56591a5ce30 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/4231506893,v1:172.21.15.115:6801/4231506893] [v2:172.21.15.115:6802/4231506893,v1:172.21.15.115:6803/4231506893] exists,up dada34c6-4dc6-4f9f-bc31-85983fd2d7fd 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6802/1995864012,v1:172.21.15.117:6803/1995864012] [v2:172.21.15.117:6804/1995864012,v1:172.21.15.117:6805/1995864012] exists,up 6c0fc051-a4a0-4ce5-989a-668e75bbe3c0 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/2054405220,v1:172.21.15.100:6801/2054405220] [v2:172.21.15.100:6802/2054405220,v1:172.21.15.100:6803/2054405220] exists,up 36a46387-b859-4937-bbd0-5dd708630c6e 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/1081990569,v1:172.21.15.107:6812/1081990569] [v2:172.21.15.107:6814/1081990569,v1:172.21.15.107:6816/1081990569] exists,up 073f3d23-77f0-4fd3-b067-27fa81137288 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 up in weight 1 up_from 22 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/1095047955,v1:172.21.15.115:6825/1095047955] [v2:172.21.15.115:6826/1095047955,v1:172.21.15.115:6827/1095047955] exists,up 8a81592f-3704-48f6-89ba-cc4997c2105e 2024-09-16T06:48:16.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6818/3607859516,v1:172.21.15.117:6819/3607859516] [v2:172.21.15.117:6820/3607859516,v1:172.21.15.117:6821/3607859516] exists,up 3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/232055141,v1:172.21.15.100:6817/232055141] [v2:172.21.15.100:6818/232055141,v1:172.21.15.100:6819/232055141] exists,up 7ce38131-057f-4d45-88a5-dae8bcceae3c 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/1511881207,v1:172.21.15.107:6825/1511881207] [v2:172.21.15.107:6826/1511881207,v1:172.21.15.107:6827/1511881207] exists,up aa7fde6e-6563-4e37-8af2-9114d0091362 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/435900412,v1:172.21.15.115:6809/435900412] [v2:172.21.15.115:6810/435900412,v1:172.21.15.115:6811/435900412] exists,up 05c5389d-8082-4be3-aa6d-138e00720f06 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6810/3208528083,v1:172.21.15.117:6811/3208528083] [v2:172.21.15.117:6812/3208528083,v1:172.21.15.117:6813/3208528083] exists,up 99552590-1d0a-420e-b0c3-357246880658 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/3107910928,v1:172.21.15.100:6825/3107910928] [v2:172.21.15.100:6826/3107910928,v1:172.21.15.100:6827/3107910928] exists,up 165fea1c-74b6-4806-a0d9-47441e161ed5 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/316232366,v1:172.21.15.107:6801/316232366] [v2:172.21.15.107:6802/316232366,v1:172.21.15.107:6803/316232366] exists,up 03402d0c-e7fd-4233-b2cd-bf79cd75e9bc 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] [v2:172.21.15.115:6818/693497656,v1:172.21.15.115:6819/693497656] exists,up 7b027faf-5c9e-4aee-9cb9-9c6459b8fe39 2024-09-16T06:48:16.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] [v2:172.21.15.117:6828/579912865,v1:172.21.15.117:6829/579912865] exists,up 4877aa40-c0d8-494f-8a98-36089ed694c2 2024-09-16T06:48:16.661 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/1028275949,v1:172.21.15.100:6809/1028275949] [v2:172.21.15.100:6810/1028275949,v1:172.21.15.100:6811/1028275949] exists,up 54494294-f25f-41f2-bff7-106492665e07', 'osd.1 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/930331367,v1:172.21.15.107:6809/930331367] [v2:172.21.15.107:6811/930331367,v1:172.21.15.107:6813/930331367] exists,up a265e752-1cd6-497e-bcb9-c56591a5ce30', 'osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/4231506893,v1:172.21.15.115:6801/4231506893] [v2:172.21.15.115:6802/4231506893,v1:172.21.15.115:6803/4231506893] exists,up dada34c6-4dc6-4f9f-bc31-85983fd2d7fd', 'osd.3 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6802/1995864012,v1:172.21.15.117:6803/1995864012] [v2:172.21.15.117:6804/1995864012,v1:172.21.15.117:6805/1995864012] exists,up 6c0fc051-a4a0-4ce5-989a-668e75bbe3c0', 'osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/2054405220,v1:172.21.15.100:6801/2054405220] [v2:172.21.15.100:6802/2054405220,v1:172.21.15.100:6803/2054405220] exists,up 36a46387-b859-4937-bbd0-5dd708630c6e', 'osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/1081990569,v1:172.21.15.107:6812/1081990569] [v2:172.21.15.107:6814/1081990569,v1:172.21.15.107:6816/1081990569] exists,up 073f3d23-77f0-4fd3-b067-27fa81137288', 'osd.6 up in weight 1 up_from 22 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/1095047955,v1:172.21.15.115:6825/1095047955] [v2:172.21.15.115:6826/1095047955,v1:172.21.15.115:6827/1095047955] exists,up 8a81592f-3704-48f6-89ba-cc4997c2105e', 'osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6818/3607859516,v1:172.21.15.117:6819/3607859516] [v2:172.21.15.117:6820/3607859516,v1:172.21.15.117:6821/3607859516] exists,up 3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27', 'osd.8 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/232055141,v1:172.21.15.100:6817/232055141] [v2:172.21.15.100:6818/232055141,v1:172.21.15.100:6819/232055141] exists,up 7ce38131-057f-4d45-88a5-dae8bcceae3c', 'osd.9 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/1511881207,v1:172.21.15.107:6825/1511881207] [v2:172.21.15.107:6826/1511881207,v1:172.21.15.107:6827/1511881207] exists,up aa7fde6e-6563-4e37-8af2-9114d0091362', 'osd.10 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/435900412,v1:172.21.15.115:6809/435900412] [v2:172.21.15.115:6810/435900412,v1:172.21.15.115:6811/435900412] exists,up 05c5389d-8082-4be3-aa6d-138e00720f06', 'osd.11 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6810/3208528083,v1:172.21.15.117:6811/3208528083] [v2:172.21.15.117:6812/3208528083,v1:172.21.15.117:6813/3208528083] exists,up 99552590-1d0a-420e-b0c3-357246880658', 'osd.12 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/3107910928,v1:172.21.15.100:6825/3107910928] [v2:172.21.15.100:6826/3107910928,v1:172.21.15.100:6827/3107910928] exists,up 165fea1c-74b6-4806-a0d9-47441e161ed5', 'osd.13 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/316232366,v1:172.21.15.107:6801/316232366] [v2:172.21.15.107:6802/316232366,v1:172.21.15.107:6803/316232366] exists,up 03402d0c-e7fd-4233-b2cd-bf79cd75e9bc', 'osd.14 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] [v2:172.21.15.115:6818/693497656,v1:172.21.15.115:6819/693497656] exists,up 7b027faf-5c9e-4aee-9cb9-9c6459b8fe39', 'osd.15 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] [v2:172.21.15.117:6828/579912865,v1:172.21.15.117:6829/579912865] exists,up 4877aa40-c0d8-494f-8a98-36089ed694c2'] 2024-09-16T06:48:16.661 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T06:48:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mon.a", 2024-09-16T06:48:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster": "ceph", 2024-09-16T06:48:16.770 INFO:teuthology.orchestra.run.smithi100.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "admin_socket_mode": "", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_client_required": "cephx, none", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_cluster_required": "cephx", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_debug": "true", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_service_required": "cephx", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_supported": "cephx", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio": "true", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T06:48:16.771 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio_reap_max": "16", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_async_discard": "false", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_block_size": "4096", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_aio": "false", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_enable_discard": "false", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_flock_retry": "3", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_inject_crash": "0", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_ioring": "false", 2024-09-16T06:48:16.772 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_ioring_hipri": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_type": "", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_allocator": "hybrid", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_buffered_io": "true", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T06:48:16.773 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_replay_recovery": "false", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_sync_write": "false", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T06:48:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_allocator": "hybrid", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_create": "true", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_db_create": "false", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_db_path": "", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_db_size": "0", 2024-09-16T06:48:16.775 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_path": "", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_size": "107374182400", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_wal_create": "false", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_wal_path": "", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs": "true", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T06:48:16.776 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_autotune": "true", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_size": "0", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_type": "2q", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T06:48:16.777 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_clone_cow": "true", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_mode": "none", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_freelist": "false", 2024-09-16T06:48:16.778 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T06:48:16.779 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fail_eio": "false", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T06:48:16.780 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T06:48:16.781 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_blob_size": "0", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T06:48:16.782 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-16T06:48:16.783 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_mem": "512", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_tracing": "false", 2024-09-16T06:48:16.784 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T06:48:16.785 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_cluster_require_version": "2", 2024-09-16T06:48:16.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_require_signatures": "false", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_require_version": "2", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_service_require_signatures": "false", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_service_require_version": "2", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_sign_messages": "true", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "chdir": "", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_acl_type": "", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_asio_thread_count": "2", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_cache_mid": "0.750000", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_cache_size": "16384", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_caps_release_delay": "5", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_check_pool_perm": "true", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T06:48:16.787 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_force_sync_read": "false", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_getattr_caps": "false", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_inject_features": "", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_die_on_failed_remount": "false", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_dirsize_rbytes": "true", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_force_lazyio": "false", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_fs": "", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_inject_release_failure": "false", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_max_inline_size": "4096", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T06:48:16.788 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mds_namespace": "", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_metadata": "", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mount_gid": "-1", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mount_timeout": "300", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mount_uid": "-1", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mountpoint": "/", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_notify_timeout": "10", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc": "true", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_max_objects": "1000", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_size": "209715200", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_permissions": "true", 2024-09-16T06:48:16.789 INFO:teuthology.orchestra.run.smithi100.stdout: "client_quota": "true", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_quota_df": "true", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_readahead_max_bytes": "0", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_readahead_max_periods": "4", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_readahead_min": "131072", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_reconnect_stale": "false", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_shutdown_timeout": "30", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_snapdir": ".snap", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_tick_interval": "1", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_trace": "", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_use_faked_inos": "false", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_use_random_mds": "false", 2024-09-16T06:48:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_graylog": "false", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_graylog_port": "12201", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_monitors": "default=true", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_syslog": "false", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_syslog_level": "info", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "-", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_network": "", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_network_interface": "", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zlib_isal": "false", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zlib_level": "5", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zstd_level": "1", 2024-09-16T06:48:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_seastar_smp": "1", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_location": "", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_location_hook": "", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_location_hook_timeout": "10", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "daemonize": "false", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "daos_pool": "tank", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-16T06:48:16.792 INFO:teuthology.orchestra.run.smithi100.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_alienstore": "0/5", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_asok": "1/5", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_asok_assert_abort": "false", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_auth": "20/20", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_bdev": "1/3", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_bluefs": "1/5", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_bluestore": "1/5", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_buffer": "0/1", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T06:48:16.793 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_cephsqlite": "0/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_client": "0/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_compressor": "1/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_context": "0/1", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_crush": "1/1", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_crypto": "1/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_cyanstore": "0/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_dpdk": "1/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_eventtrace": "1/5", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_filer": "0/1", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_filestore": "1/3", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_finisher": "1/1", 2024-09-16T06:48:16.794 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_fuse": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_javaclient": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_journal": "1/3", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_journaler": "0/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_kstore": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_leveldb": "4/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_lockdep": "0/1", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mclock": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_balancer": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_locker": "1/5", 2024-09-16T06:48:16.795 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_log": "1/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_migrator": "1/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_memstore": "1/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mgr": "2/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mgrc": "1/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mon": "20/20", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_monc": "0/10", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_ms": "1/1", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_none": "0/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_objclass": "0/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_objectcacher": "0/5", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_objecter": "0/1", 2024-09-16T06:48:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_optracker": "0/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_osd": "1/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_paxos": "20/20", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_perfcounter": "1/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_prioritycache": "1/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rados": "0/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd": "0/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd_replay": "0/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_refs": "0/0", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_reserver": "1/1", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw": "1/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_access": "1/5", 2024-09-16T06:48:16.797 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_dbstore": "1/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_flight": "1/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_sync": "1/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rocksdb": "4/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_backref": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_cache": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_device": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_epm": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_journal": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_lba": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_odata": "0/5", 2024-09-16T06:48:16.798 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_omap": "0/5", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_onode": "0/5", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_t": "0/5", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_tm": "0/5", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_striper": "0/1", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_test": "0/5", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_throttle": "1/1", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_timer": "0/1", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_tp": "0/5", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "device_failure_prediction_mode": "none", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-16T06:48:16.799 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_graylog": "false", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_journald": "false", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_stderr": "true", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_syslog": "false", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event_tracing": "false", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_http_port": "9926", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_prio_limit": "5", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_sort_metrics": "true", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_stats_period": "5", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "fake_statfs_for_testing": "0", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "fatal_signal_handlers": "true", 2024-09-16T06:48:16.800 INFO:teuthology.orchestra.run.smithi100.stdout: "filer_max_purge_ops": "10", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filer_max_truncate_ops": "128", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_blackhole": "false", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_btrfs_snap": "true", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_caller_concurrency": "10", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_omap_check": "false", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_verify_split": "false", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_dump_file": "", 2024-09-16T06:48:16.801 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fadvise": "true", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fail_eio": "true", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fd_cache_size": "128", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fiemap": "false", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_inject_stall": "0", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_journal_parallel": "false", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_journal_trailing": "false", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_journal_writeahead": "false", 2024-09-16T06:48:16.802 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_kill_at": "0", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T06:48:16.803 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_merge_threshold": "-10", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_odsync_write": "false", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_omap_backend_path": "", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_op_threads": "2", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_punch_hole": "false", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T06:48:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_ops": "50", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_seek_data_hole": "false", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_sloppy_crc": "false", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_splice": "false", 2024-09-16T06:48:16.805 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_split_multiple": "2", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_split_rand_factor": "20", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_update_to": "1000", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T06:48:16.806 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_xfs_extsize": "false", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_zfs_snap": "false", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fio_dir": "/tmp/fio", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "b2976256-637c-4e28-94f6-99f20017bc5e", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_allow_other": "true", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_big_writes": "true", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_debug": "false", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_default_permissions": "false", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_disable_pagecache": "false", 2024-09-16T06:48:16.807 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_max_write": "0", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_multithreaded": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_require_active_mds": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_set_user_groups": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_splice_move": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_splice_read": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_splice_write": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "gss_target_name": "ceph", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_file": "", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_inject_failure": "0", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_interval": "5", 2024-09-16T06:48:16.808 INFO:teuthology.orchestra.run.smithi100.stdout: "host": "smithi100", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T06:48:16.809 INFO:teuthology.orchestra.run.smithi100.stdout: "inject_early_sigterm": "false", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "jaeger_agent_port": "6799", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "jaeger_tracing_enable": "false", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_aio": "true", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_align_min_size": "65536", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_block_align": "true", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_block_size": "4096", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_dio": "true", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_discard": "false", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_force_aio": "false", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_ignore_corruption": "false", 2024-09-16T06:48:16.810 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_max_write_entries": "100", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_replay_from": "0", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_write_header_frequency": "0", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_zero_on_create": "false", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journaler_prefetch_periods": "10", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journaler_prezero_periods": "5", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "journaler_write_head_interval": "15", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "", 2024-09-16T06:48:16.811 INFO:teuthology.orchestra.run.smithi100.stdout: "keyfile": "", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_backend": "rocksdb", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_max_bytes": "67108864", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_max_ops": "512", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_onode_map_size": "1024", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_sync_transaction": "false", 2024-09-16T06:48:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_block_size": "65536", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_bloom_size": "0", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_cache_size": "536870912", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_compact_on_mount": "false", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_compression": "false", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_log": "", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_max_open_files": "0", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_paranoid": "false", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "librados_thread_count": "2", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "lockdep": "false", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "lockdep_force_backtrace": "false", 2024-09-16T06:48:16.813 INFO:teuthology.orchestra.run.smithi100.stdout: "log_coarse_timestamps": "true", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_flush_on_exit": "false", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_graylog_port": "12201", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_max_new": "1000", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_max_recent": "10000", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_stderr_prefix": "", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": "true", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_graylog": "false", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_journald": "false", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_stderr": "false", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_syslog": "false", 2024-09-16T06:48:16.814 INFO:teuthology.orchestra.run.smithi100.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_action_on_write_error": "1", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_alternate_name_max": "8192", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_asio_thread_count": "2", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_export_pin": "true", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_interval": "10", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_max": "-1", 2024-09-16T06:48:16.815 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_max_until": "-1", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_merge_size": "50", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_mode": "0", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_bits": "3", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_size": "10000", 2024-09-16T06:48:16.816 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_mid": "0.700000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_trim_interval": "1", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T06:48:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_auth_pins": "false", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_frag": "false", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_scatterstat": "false", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_subtrees": "false", 2024-09-16T06:48:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_default_dir_hash": "2", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_defer_session_stale": "true", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_max_entries": "0", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_prefetch": "true", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_early_reply": "true", 2024-09-16T06:48:16.819 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_enable_op_tracker": "true", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_enforce_unique_name": "true", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T06:48:16.820 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_health_dummy": "false", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_join_fs": "", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_journal_format": "1", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_create_at": "0", 2024-09-16T06:48:16.821 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_export_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_import_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_journal_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_link_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_openc_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_rename_at": "0", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_max_events": "-1", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_max_segments": "128", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_pause": "false", 2024-09-16T06:48:16.822 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_segment_size": "0", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_completed_requests": "100000", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_export_size": "20971520", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_file_recover": "32", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_purge_files": "64", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_purge_ops": "8192", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T06:48:16.823 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_metrics_update_interval": "2", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_min_caps_per_client": "100", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_numa_node": "-1", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_duration": "600", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_size": "20", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_log_threshold": "5", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_ping_grace": "15", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_ping_interval": "5", 2024-09-16T06:48:16.824 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_max_caps": "30000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_replay_interval": "1.000000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_root_ino_gid": "0", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_root_ino_uid": "0", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T06:48:16.825 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_shutdown_check": "0", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_skip_ino": "0", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_snap_min_uid": "0", 2024-09-16T06:48:16.826 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_snap_rstat": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_symlink_recovery": "true", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_thrash_exports": "0", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_thrash_fragments": "0", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_tick_interval": "5.000000", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_valgrind_exit": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_verify_backtrace": "1", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_verify_scatter": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_wipe_sessions": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool_debug": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T06:48:16.827 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_page_set": "false", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_page_size": "65536", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_client_bytes": "134217728", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_client_messages": "512", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_disabled_modules": "", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T06:48:16.828 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mds_messages": "128", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mon_messages": "128", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_osd_messages": "8192", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_pool": "true", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_standby_modules": "true", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_stats_period": "5", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_stats_threshold": "5", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_tick_period": "2", 2024-09-16T06:48:16.829 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_allow_pool_delete": "true", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_bytes": "104857600", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T06:48:16.830 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_file_level": "debug", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T06:48:16.831 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_compact_on_start": "false", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_compact_on_trim": "true", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cpu_threads": "4", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T06:48:16.832 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data_avail_crit": "5", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data_avail_warn": "5", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_dump_json": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_extra_checks": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T06:48:16.833 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_require_quincy": "false", 2024-09-16T06:48:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_require_reef": "false", 2024-09-16T06:48:16.835 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_election_default_strategy": "3", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_election_timeout": "5.000000", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_enable_op_tracker": "true", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_fake_pool_delete": "false", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_force_quorum_join": "false", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T06:48:16.836 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_log_update_period": "5", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_max_detail": "50", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_to_clog": "true", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_host": "172.21.15.100,172.21.15.107,172.21.15.115", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_host_override": "", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_initial_members": "", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T06:48:16.837 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_lease": "5.000000", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_log_full_interval": "50", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_log_max": "10000", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_log_max_summary": "50", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_log_epochs": "500", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_osd": "10000", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T06:48:16.838 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_memory_autotune": "true", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_memory_target": "2147483648", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_digest_period": "5", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_complaint_time": "30", 2024-09-16T06:48:16.839 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_duration": "600", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_size": "20", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_log_threshold": "5", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_cache_size": "500", 2024-09-16T06:48:16.840 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T06:48:16.841 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_report_timeout": "900", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T06:48:16.842 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_max_osds": "4", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T06:48:16.843 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_interval": "300", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_max_keys": "100", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_timeout": "300", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_session_timeout": "300", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_smart_report_timeout": "5", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_pool_size": "4", 2024-09-16T06:48:16.844 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_debug": "false", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_tick_interval": "5", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_use_min_delay_socket": "false", 2024-09-16T06:48:16.845 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_older_version": "true", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T06:48:16.846 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "monmap": "", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-16T06:48:16.847 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_tracing_enabled": "false", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_op_threads": "3", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_cm": "false", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_device_name": "", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T06:48:16.848 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_sl": "3", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_type": "ib", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_reap_threshold": "5", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_before_connect": "false", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_ipv4": "true", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_ipv6": "false", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_msgr1": "true", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_msgr2": "true", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_port_max": "7568", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_port_min": "6800", 2024-09-16T06:48:16.849 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_retry_count": "3", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_retry_delay": "5", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_client": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_mds": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_mgr": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_mon": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_osd": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_client_mode": "crc secure", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_cluster_type": "", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_compress_secure": "false", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T06:48:16.850 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_crc_data": "true", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_crc_header": "true", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_bug": "true", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_old_message": "true", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T06:48:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_hugepages": "", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_lro": "true", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_pmd": "", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_port_id": "0", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T06:48:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dump_on_send": "false", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_delay_probability": "0.005000", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_delay_type": "osd", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_internal_delays": "0.002000", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_network_congestion": "0", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_socket_failures": "2500", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_max_accept_failures": "4", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_max_backoff": "15.000000", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T06:48:16.853 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_osd_compress_mode": "none", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_pq_min_cost": "65536", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_public_type": "", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_service_mode": "crc secure", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_nodelay": "true", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_type": "async+posix", 2024-09-16T06:48:16.854 INFO:teuthology.orchestra.run.smithi100.stdout: "no_config_file": "false", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_inflight_ops": "1024", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_timeout": "10.000000", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "objectstore_blackhole": "false", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "openssl_engine_opts": "", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_max_ops": "4", 2024-09-16T06:48:16.855 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_slop": "0.020000", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_asio_thread_count": "2", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backfill_scan_max": "512", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backfill_scan_min": "64", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backoff_on_peering": "false", 2024-09-16T06:48:16.856 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_beacon_report_interval": "300", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_duration": "30", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_blkin_trace_all": "false", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T06:48:16.857 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_update_on_start": "true", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_message_cap": "256", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_op_priority": "63", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_watch_timeout": "30", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_command_max_records": "256", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_compact_on_start": "false", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_update_on_start": "true", 2024-09-16T06:48:16.858 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_op_order": "false", 2024-09-16T06:48:16.859 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_shutdown": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T06:48:16.860 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_default_notify_timeout": "30", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_enable_op_tracker": "true", 2024-09-16T06:48:16.861 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_info": "true", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_shutdown": "true", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_function_tracing": "false", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_grace": "20", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_interval": "6", 2024-09-16T06:48:16.862 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_stale": "600", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T06:48:16.863 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_journal_size": "5120", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_kill_backfill_at": "0", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_cache_size": "50", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_dedup": "true", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_message_max": "40", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_attr_name_len": "100", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_attr_size": "0", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_backfills": "1", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_markdown_count": "5", 2024-09-16T06:48:16.864 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_markdown_period": "600", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_object_name_len": "2048", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_object_size": "134217728", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pgls": "1024", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_push_cost": "8388608", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_push_objects": "10", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_scrubs": "3", 2024-09-16T06:48:16.865 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_write_size": "90", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_profile": "balanced", 2024-09-16T06:48:16.866 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_base": "805306368", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T06:48:16.867 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_target": "4294967296", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_target_autotune": "false", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_min_recovery_priority": "0", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_report_interval": "5", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_num_cache_shards": "32", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T06:48:16.868 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_numa_node": "-1", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objecter_finishers": "1", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objectstore": "bluestore", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objectstore_fuse": "false", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objectstore_tracing": "false", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_duration": "600", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_size": "20", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_log_threshold": "5", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_shards": "0", 2024-09-16T06:48:16.869 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_thread_timeout": "15", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_open_classes_on_start": "true", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_os_flags": "0", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_peering_op_priority": "255", 2024-09-16T06:48:16.870 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_delete_priority": "5", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T06:48:16.871 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_crimson": "false", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flags": "0", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T06:48:16.872 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_min_size": "0", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_size": "2", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_type": "replicated", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_cost": "20971520", 2024-09-16T06:48:16.873 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_active": "10", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_single_start": "10", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_op_priority": "3", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_priority": "5", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T06:48:16.874 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_repair_during_recovery": "false", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_cost": "52428800", 2024-09-16T06:48:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_end_hour": "0", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_priority": "5", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-16T06:48:16.876 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_skip_data_digest": "false", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_smart_report_timeout": "5", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_priority": "5", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T06:48:16.877 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_target_transaction_size": "30", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T06:48:16.878 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tracing": "false", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_use_stale_snap": "false", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_kill_at": "0", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_max_join_drift": "10", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_min": "500", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_min_wait": "0.050000", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_service_trim_max": "500", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_service_trim_min": "250", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_stash_full_interval": "25", 2024-09-16T06:48:16.879 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_trim_max": "500", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_trim_min": "250", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "perf": "true", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "-", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrv": "", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "public_bind_addr": "-", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "public_network": "", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "public_network_interface": "", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "qat_compressor_enabled": "false", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "qat_compressor_session_max_number": "256", 2024-09-16T06:48:16.880 INFO:teuthology.orchestra.run.smithi100.stdout: "rados_mon_op_timeout": "0", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rados_osd_op_timeout": "0", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rados_tracing": "false", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_atime_update_interval": "60", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache": "true", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T06:48:16.881 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_policy": "writearound", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_size": "33554432", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_compression_hint": "none", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_clone_format": "auto", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_data_pool": "", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_features": "61", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_format": "2", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_map_options": "", 2024-09-16T06:48:16.882 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_order": "22", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_pool": "rbd", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_stripe_count": "0", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_io_scheduler": "simple", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T06:48:16.883 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_order": "24", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_pool": "", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_splay_width": "4", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T06:48:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T06:48:16.885 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T06:48:16.886 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_op_threads": "1", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_plugins": "", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T06:48:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T06:48:16.888 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_tracing": "false", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_validate_names": "true", 2024-09-16T06:48:16.889 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_validate_pool": "true", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "restapi_base_url": "", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "restapi_log_level": "", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_admin_entry": "admin", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_backend_store": "rados", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_barbican_url": "", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_beast_enable_async": "true", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T06:48:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cache_enabled": "true", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cache_lru_size": "10000", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_config_store": "rados", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_content_length_compat": "false", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cross_domain_policy": "", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T06:48:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T06:48:16.892 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T06:48:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T06:48:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_window": "30", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T06:48:16.895 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T06:48:16.896 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dns_name": "", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dns_s3website_name": "", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_ops_log": "false", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T06:48:16.897 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_static_website": "false", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_usage_log": "false", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_expose_bucket": "false", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_extended_http_attrs": "", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_filter": "none", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_objs": "32", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T06:48:16.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_init_timeout": "300", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T06:48:16.899 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_password": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_project": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_token": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_user": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_api_version": "2", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T06:48:16.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_url": "", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_objs": "32", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_worker": "3", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T06:48:16.901 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_http_headers": "", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T06:48:16.902 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_attr_size": "0", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_listing_results": "1000", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_notify_retries": "10", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T06:48:16.903 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T06:48:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_num_control_oids": "8", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_numa_node": "-1", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T06:48:16.905 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_op_tracing": "false", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_opa_token": "", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_opa_url": "", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_rados": "false", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T06:48:16.906 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_print_continue": "true", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rados_tracing": "false", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_realm": "", 2024-09-16T06:48:16.907 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_realm_id": "", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_region": "", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_request_uri": "", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T06:48:16.908 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_resolve_cname": "false", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_run_sync_thread": "true", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_script_uri": "", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_service_provider_name": "", 2024-09-16T06:48:16.909 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_client_id": "", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_client_secret": "", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_entry": "sts", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_key": "sts", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_auth_url": "", 2024-09-16T06:48:16.910 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_custom_header": "", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_need_stats": "true", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_tenant_name": "", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_url": "", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_lease_period": "120", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T06:48:16.911 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_thread_pool_size": "512", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_comment": "", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_createby": "", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_encoding": "", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_flag": "false", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_origin": "", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T06:48:16.912 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_tracker": "", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_max_shards": "32", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_use_opa_authz": "false", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_policies_max_num": "100", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T06:48:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_unique_email": "true", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_verify_ssl": "true", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zone": "", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zone_id": "", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zonegroup": "", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zonegroup_id": "", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgwlc_auto_session_clear": "true", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_block_size": "4096", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T06:48:16.914 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T06:48:16.915 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_partition_filters": "false", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_perf": "false", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "run_dir": "/var/run/ceph", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_block_create": "true", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_device_size": "53687091200", 2024-09-16T06:48:16.916 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_main_device_type": "SSD", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_segment_size": "67108864", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "set_keepcaps": "false", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "setgroup": "", 2024-09-16T06:48:16.917 INFO:teuthology.orchestra.run.smithi100.stdout: "setuser": "", 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "setuser_match_path": "", 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "thp": "false", 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "threadpool_default_timeout": "60", 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout: "throttler_perf_counter": "true" 2024-09-16T06:48:16.918 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:16.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T06:48:17.064 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a: {} 2024-09-16T06:48:17.065 INFO:teuthology.orchestra.run.smithi100.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T06:48:17.081 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b: {} 2024-09-16T06:48:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T06:48:17.099 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c: {} 2024-09-16T06:48:17.099 INFO:teuthology.orchestra.run.smithi100.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T06:48:17.114 DEBUG:teuthology.orchestra.run.smithi115:> type ceph-objectstore-tool 2024-09-16T06:48:17.119 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T06:48:17.120 DEBUG:teuthology.orchestra.run.smithi100:> type ceph-objectstore-tool 2024-09-16T06:48:17.123 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T06:48:17.124 DEBUG:teuthology.orchestra.run.smithi107:> type ceph-objectstore-tool 2024-09-16T06:48:17.129 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T06:48:17.129 DEBUG:teuthology.orchestra.run.smithi117:> type ceph-objectstore-tool 2024-09-16T06:48:17.135 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T06:48:17.135 INFO:teuthology.run_tasks:Running task rados... 2024-09-16T06:48:17.147 INFO:tasks.rados:Beginning rados... 2024-09-16T06:48:17.147 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-16T06:48:17.147 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}} 2024-09-16T06:48:17.147 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-16T06:48:17.148 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-16T06:48:17.179 INFO:tasks.rados:joining rados 2024-09-16T06:48:17.180 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T06:48:17.180 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:17.180 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:17.180 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T06:48:17.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T06:48:17.181 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T06:48:17.181 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T06:48:17.181 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:17.181 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T06:48:17.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:48:17.182 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T06:48:17.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:48:17.183 INFO:tasks.rados:clients are ['client.0'] 2024-09-16T06:48:17.183 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-09-16T06:48:17.184 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:17.180+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:17.283 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:17.279+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:17.329 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:17.330 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:17.354 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:17.354 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:17.365 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:17.365 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:17.378 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:17.379 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:17.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:48:17.385 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:17.385+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:17.403 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:17.404 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:17.438 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:17.439 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:17.476 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:17.476 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:17.485 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:17.485+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:17.493 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:17.494 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:17.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:48:17.513 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:17.514 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 29, 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "b2976256-637c-4e28-94f6-99f20017bc5e", 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "created": "2024-09-16T06:47:53.157733+0000", 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "modified": "2024-09-16T06:48:16.797900+0000", 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "last_up_change": "2024-09-16T06:48:06.616578+0000", 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "last_in_change": "2024-09-16T06:47:58.043572+0000", 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T06:48:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_num": 5799936, 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_set": [ 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "pglog_hardlimit", 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "purged_snapdirs", 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_deletes", 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "sortbitwise" 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_version": 5, 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_snapshot": "", 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_max": 2, 2024-09-16T06:48:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd": 16, 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "require_min_compat_client": "luminous", 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "min_compat_client": "jewel", 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "require_osd_release": "reef", 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_crimson": false, 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "pools": [ 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 1, 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": ".mgr", 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:05.838641+0000", 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 1, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 1, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 1, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 1, 2024-09-16T06:48:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 1, 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "25", 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_max": 32, 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 1 2024-09-16T06:48:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": {} 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 15.380000114440918, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 15.380000114440918, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.12999999523162842, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-16T06:48:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "rbd", 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:08.489111+0000", 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 8, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 8, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 8, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 8, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 8, 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "28", 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 3, 2024-09-16T06:48:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 28, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd": {} 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 4.0199999809265137, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 4.0199999809265137, 2024-09-16T06:48:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.56000000238418579, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2.25, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2.25, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "osds": [ 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "54494294-f25f-41f2-bff7-106492665e07", 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6808", 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6809", 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6810", 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6811", 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6814", 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6815", 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6812", 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6813", 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6809/1028275949", 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6811/1028275949", 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6815/1028275949", 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6813/1028275949", 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "a265e752-1cd6-497e-bcb9-c56591a5ce30", 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6808", 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6809", 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6811", 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6813", 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6819", 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6821", 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6815", 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6817", 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6809/930331367", 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6813/930331367", 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6821/930331367", 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6817/930331367", 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "dada34c6-4dc6-4f9f-bc31-85983fd2d7fd", 2024-09-16T06:48:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6800", 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6801", 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6802", 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6803", 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.553 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6806", 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6807", 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.554 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6804", 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6805", 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6801/4231506893", 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6803/4231506893", 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6807/4231506893", 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6805/4231506893", 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "6c0fc051-a4a0-4ce5-989a-668e75bbe3c0", 2024-09-16T06:48:17.556 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.557 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6802", 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6803", 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.558 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6804", 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6805", 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6808", 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6809", 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.560 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6806", 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6807", 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6803/1995864012", 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6805/1995864012", 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6809/1995864012", 2024-09-16T06:48:17.561 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6807/1995864012", 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "36a46387-b859-4937-bbd0-5dd708630c6e", 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.562 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-09-16T06:48:17.563 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6802", 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.564 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6803", 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6806", 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6807", 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6804", 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6805", 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6801/2054405220", 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6803/2054405220", 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6807/2054405220", 2024-09-16T06:48:17.567 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6805/2054405220", 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "073f3d23-77f0-4fd3-b067-27fa81137288", 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.568 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6810", 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6812", 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6814", 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6816", 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6822", 2024-09-16T06:48:17.571 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6823", 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6818", 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6820", 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6812/1081990569", 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6816/1081990569", 2024-09-16T06:48:17.573 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6823/1081990569", 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6820/1081990569", 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "8a81592f-3704-48f6-89ba-cc4997c2105e", 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.574 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 23, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6824", 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.575 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6825", 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6826", 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6827", 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6830", 2024-09-16T06:48:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6831", 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6828", 2024-09-16T06:48:17.578 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6829", 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6825/1095047955", 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6827/1095047955", 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6831/1095047955", 2024-09-16T06:48:17.579 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6829/1095047955", 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27", 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.580 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6818", 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6819", 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6820", 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6821", 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6824", 2024-09-16T06:48:17.583 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6825", 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.584 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6822", 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6823", 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6819/3607859516", 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6821/3607859516", 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6825/3607859516", 2024-09-16T06:48:17.585 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6823/3607859516", 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7ce38131-057f-4d45-88a5-dae8bcceae3c", 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.586 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6816", 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6817", 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6818", 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6819", 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.589 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6822", 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6823", 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6820", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6821", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6817/232055141", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6819/232055141", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6823/232055141", 2024-09-16T06:48:17.591 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6821/232055141", 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "aa7fde6e-6563-4e37-8af2-9114d0091362", 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.592 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6824", 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6825", 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6826", 2024-09-16T06:48:17.594 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6827", 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6830", 2024-09-16T06:48:17.595 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6831", 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.596 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6828", 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6829", 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6825/1511881207", 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6827/1511881207", 2024-09-16T06:48:17.597 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6831/1511881207", 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6829/1511881207", 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "05c5389d-8082-4be3-aa6d-138e00720f06", 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.598 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6808", 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6809", 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6810", 2024-09-16T06:48:17.600 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6811", 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.601 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6814", 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6815", 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6812", 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6813", 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6809/435900412", 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6811/435900412", 2024-09-16T06:48:17.603 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6815/435900412", 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6813/435900412", 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "99552590-1d0a-420e-b0c3-357246880658", 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.604 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6810", 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6811", 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6812", 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6813", 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6816", 2024-09-16T06:48:17.607 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6817", 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6814", 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6815", 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6811/3208528083", 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6813/3208528083", 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6817/3208528083", 2024-09-16T06:48:17.609 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6815/3208528083", 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "165fea1c-74b6-4806-a0d9-47441e161ed5", 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.610 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6824", 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6825", 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.612 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6826", 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6827", 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.613 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6830", 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6831", 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6828", 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6829", 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6825/3107910928", 2024-09-16T06:48:17.615 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6827/3107910928", 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6831/3107910928", 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6829/3107910928", 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "03402d0c-e7fd-4233-b2cd-bf79cd75e9bc", 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.616 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6800", 2024-09-16T06:48:17.617 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6801", 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.618 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6802", 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6803", 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6806", 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6807", 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6804", 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6805", 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.621 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6801/316232366", 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6803/316232366", 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6807/316232366", 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6805/316232366", 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7b027faf-5c9e-4aee-9cb9-9c6459b8fe39", 2024-09-16T06:48:17.622 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.623 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6816", 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6817", 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.624 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6818", 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6819", 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.625 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6822", 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6823", 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.626 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6820", 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6821", 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6817/693497656", 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6819/693497656", 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6823/693497656", 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6821/693497656", 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:17.628 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "4877aa40-c0d8-494f-8a98-36089ed694c2", 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.629 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6826", 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6827", 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.630 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6828", 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6829", 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6832", 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6833", 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:17.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6830", 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6831", 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6827/579912865", 2024-09-16T06:48:17.633 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6829/579912865", 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6833/579912865", 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6831/579912865", 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_xinfo": [ 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.584963+0000", 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.635 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.677023+0000", 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.608639+0000", 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.384489+0000", 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.637 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.480551+0000", 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.638 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.476690+0000", 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.615096+0000", 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.581224+0000", 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.597357+0000", 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:17.641 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.549502+0000", 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.642 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.448485+0000", 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.495825+0000", 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.628407+0000", 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:17.644 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.368452+0000", 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.645 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.600045+0000", 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.567594+0000", 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:17.646 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap": [], 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_items": [], 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_primaries": [], 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_temp": [], 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_temp": [], 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklist": {}, 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "range_blocklist": {}, 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profiles": { 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "default": { 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "2", 2024-09-16T06:48:17.647 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "1", 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "jerasure", 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "reed_sol_van" 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps_queue": [], 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "new_removed_snaps": [], 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "new_purged_snaps": [ 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 2, 2024-09-16T06:48:17.648 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_node_flags": {}, 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "device_class_flags": {}, 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode": { 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_enabled": false, 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_bucket_count": 0, 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "degraded_stretch_mode": 0, 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "recovering_stretch_mode": 0, 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_bucket": 0 2024-09-16T06:48:17.649 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:17.650 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:17.651 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:17.651 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:17.653 INFO:tasks.thrashosds.thrasher:j is {'epoch': 29, 'fsid': 'b2976256-637c-4e28-94f6-99f20017bc5e', 'created': '2024-09-16T06:47:53.157733+0000', 'modified': '2024-09-16T06:48:16.797900+0000', 'last_up_change': '2024-09-16T06:48:06.616578+0000', 'last_in_change': '2024-09-16T06:47:58.043572+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T06:48:05.838641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T06:48:08.489111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '54494294-f25f-41f2-bff7-106492665e07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6808', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6809', 'nonce': 1028275949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6810', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6811', 'nonce': 1028275949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6814', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6815', 'nonce': 1028275949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6812', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6813', 'nonce': 1028275949}]}, 'public_addr': '172.21.15.100:6809/1028275949', 'cluster_addr': '172.21.15.100:6811/1028275949', 'heartbeat_back_addr': '172.21.15.100:6815/1028275949', 'heartbeat_front_addr': '172.21.15.100:6813/1028275949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a265e752-1cd6-497e-bcb9-c56591a5ce30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 930331367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6811', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 930331367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6819', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 930331367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6815', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 930331367}]}, 'public_addr': '172.21.15.107:6809/930331367', 'cluster_addr': '172.21.15.107:6813/930331367', 'heartbeat_back_addr': '172.21.15.107:6821/930331367', 'heartbeat_front_addr': '172.21.15.107:6817/930331367', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'dada34c6-4dc6-4f9f-bc31-85983fd2d7fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 4231506893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 4231506893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 4231506893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 4231506893}]}, 'public_addr': '172.21.15.115:6801/4231506893', 'cluster_addr': '172.21.15.115:6803/4231506893', 'heartbeat_back_addr': '172.21.15.115:6807/4231506893', 'heartbeat_front_addr': '172.21.15.115:6805/4231506893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6c0fc051-a4a0-4ce5-989a-668e75bbe3c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6802', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6803', 'nonce': 1995864012}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6804', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6805', 'nonce': 1995864012}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6808', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6809', 'nonce': 1995864012}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6806', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6807', 'nonce': 1995864012}]}, 'public_addr': '172.21.15.117:6803/1995864012', 'cluster_addr': '172.21.15.117:6805/1995864012', 'heartbeat_back_addr': '172.21.15.117:6809/1995864012', 'heartbeat_front_addr': '172.21.15.117:6807/1995864012', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '36a46387-b859-4937-bbd0-5dd708630c6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 2054405220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6802', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6803', 'nonce': 2054405220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6806', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6807', 'nonce': 2054405220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6804', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6805', 'nonce': 2054405220}]}, 'public_addr': '172.21.15.100:6801/2054405220', 'cluster_addr': '172.21.15.100:6803/2054405220', 'heartbeat_back_addr': '172.21.15.100:6807/2054405220', 'heartbeat_front_addr': '172.21.15.100:6805/2054405220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '073f3d23-77f0-4fd3-b067-27fa81137288', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6812', 'nonce': 1081990569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6816', 'nonce': 1081990569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 1081990569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6820', 'nonce': 1081990569}]}, 'public_addr': '172.21.15.107:6812/1081990569', 'cluster_addr': '172.21.15.107:6816/1081990569', 'heartbeat_back_addr': '172.21.15.107:6823/1081990569', 'heartbeat_front_addr': '172.21.15.107:6820/1081990569', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8a81592f-3704-48f6-89ba-cc4997c2105e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 1095047955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1095047955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1095047955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1095047955}]}, 'public_addr': '172.21.15.115:6825/1095047955', 'cluster_addr': '172.21.15.115:6827/1095047955', 'heartbeat_back_addr': '172.21.15.115:6831/1095047955', 'heartbeat_front_addr': '172.21.15.115:6829/1095047955', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6818', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6819', 'nonce': 3607859516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6820', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6821', 'nonce': 3607859516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6824', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6825', 'nonce': 3607859516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6822', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6823', 'nonce': 3607859516}]}, 'public_addr': '172.21.15.117:6819/3607859516', 'cluster_addr': '172.21.15.117:6821/3607859516', 'heartbeat_back_addr': '172.21.15.117:6825/3607859516', 'heartbeat_front_addr': '172.21.15.117:6823/3607859516', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '7ce38131-057f-4d45-88a5-dae8bcceae3c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6816', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6817', 'nonce': 232055141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6818', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6819', 'nonce': 232055141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6822', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6823', 'nonce': 232055141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6820', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6821', 'nonce': 232055141}]}, 'public_addr': '172.21.15.100:6817/232055141', 'cluster_addr': '172.21.15.100:6819/232055141', 'heartbeat_back_addr': '172.21.15.100:6823/232055141', 'heartbeat_front_addr': '172.21.15.100:6821/232055141', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'aa7fde6e-6563-4e37-8af2-9114d0091362', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 1511881207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 1511881207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 1511881207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 1511881207}]}, 'public_addr': '172.21.15.107:6825/1511881207', 'cluster_addr': '172.21.15.107:6827/1511881207', 'heartbeat_back_addr': '172.21.15.107:6831/1511881207', 'heartbeat_front_addr': '172.21.15.107:6829/1511881207', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '05c5389d-8082-4be3-aa6d-138e00720f06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 435900412}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 435900412}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 435900412}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 435900412}]}, 'public_addr': '172.21.15.115:6809/435900412', 'cluster_addr': '172.21.15.115:6811/435900412', 'heartbeat_back_addr': '172.21.15.115:6815/435900412', 'heartbeat_front_addr': '172.21.15.115:6813/435900412', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '99552590-1d0a-420e-b0c3-357246880658', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6810', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6811', 'nonce': 3208528083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6812', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6813', 'nonce': 3208528083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6816', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6817', 'nonce': 3208528083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6814', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6815', 'nonce': 3208528083}]}, 'public_addr': '172.21.15.117:6811/3208528083', 'cluster_addr': '172.21.15.117:6813/3208528083', 'heartbeat_back_addr': '172.21.15.117:6817/3208528083', 'heartbeat_front_addr': '172.21.15.117:6815/3208528083', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '165fea1c-74b6-4806-a0d9-47441e161ed5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6824', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6825', 'nonce': 3107910928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6826', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6827', 'nonce': 3107910928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6830', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6831', 'nonce': 3107910928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6828', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6829', 'nonce': 3107910928}]}, 'public_addr': '172.21.15.100:6825/3107910928', 'cluster_addr': '172.21.15.100:6827/3107910928', 'heartbeat_back_addr': '172.21.15.100:6831/3107910928', 'heartbeat_front_addr': '172.21.15.100:6829/3107910928', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '03402d0c-e7fd-4233-b2cd-bf79cd75e9bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 316232366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 316232366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 316232366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 316232366}]}, 'public_addr': '172.21.15.107:6801/316232366', 'cluster_addr': '172.21.15.107:6803/316232366', 'heartbeat_back_addr': '172.21.15.107:6807/316232366', 'heartbeat_front_addr': '172.21.15.107:6805/316232366', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '7b027faf-5c9e-4aee-9cb9-9c6459b8fe39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 693497656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 693497656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 693497656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 693497656}]}, 'public_addr': '172.21.15.115:6817/693497656', 'cluster_addr': '172.21.15.115:6819/693497656', 'heartbeat_back_addr': '172.21.15.115:6823/693497656', 'heartbeat_front_addr': '172.21.15.115:6821/693497656', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '4877aa40-c0d8-494f-8a98-36089ed694c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6826', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6827', 'nonce': 579912865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6828', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6829', 'nonce': 579912865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 579912865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6830', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6831', 'nonce': 579912865}]}, 'public_addr': '172.21.15.117:6827/579912865', 'cluster_addr': '172.21.15.117:6829/579912865', 'heartbeat_back_addr': '172.21.15.117:6833/579912865', 'heartbeat_front_addr': '172.21.15.117:6831/579912865', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.584963+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.677023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.608639+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.384489+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.480551+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.476690+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.615096+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.581224+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.597357+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.549502+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.448485+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.495825+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.628407+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.368452+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.600045+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.567594+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T06:48:17.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:48:17.653 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:17.654 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:17.654 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:17.654 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:17.655 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:17.655 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:17.655 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:17.655 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:17.656 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:17.649+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:17.656 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:17.657 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:17.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:48:17.657 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:17.658 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:17.681 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:17.681 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:17.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:17.708+0000 7ff24172e640 0 -- 172.21.15.100:0/1904087807 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7ff23c220c50 msgr2=0x7ff23c223060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:17.752 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:17.753+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:17.787 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:17.787 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:17.801 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:48:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:load: shec 2024-09-16T06:48:17.844 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:48:17.852 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:17.853+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:17.863 INFO:tasks.rados:starting run 0 out of 1 2024-09-16T06:48:17.863 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-16T06:48:17.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-09-16T06:48:17.923 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:17.923 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:17.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:48:17.953 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:17.953+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:17.967 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:17.967 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:48:17.985 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T06:48:17.845265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":438480,"kb_used_data":5704,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509510960,"statfs":{"total":1546188226560,"available":1545739223040,"internally_reserved":0,"allocated":5840896,"data_stored":2971066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":442473568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":16,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.099077"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:13.218482+0000","last_change":"2024-09-16T06:48:13.218911+0000","last_active":"2024-09-16T06:48:13.218482+0000","last_peered":"2024-09-16T06:48:13.218482+0000","last_clean":"2024-09-16T06:48:13.218482+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:48:13.218482+0000","last_undegraded":"2024-09-16T06:48:13.218482+0000","last_fullsized":"2024-09-16T06:48:13.218482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:09.738021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-16T06:48:16.976023+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:48:16.976023+0000","last_peered":"2024-09-16T06:48:16.976023+0000","last_clean":"2024-09-16T06:48:16.976023+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:48:16.976023+0000","last_undegraded":"2024-09-16T06:48:16.976023+0000","last_fullsized":"2024-09-16T06:48:16.976023+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.416727+0000","last_change":"2024-09-16T06:48:12.417012+0000","last_active":"2024-09-16T06:48:12.416727+0000","last_peered":"2024-09-16T06:48:12.416727+0000","last_clean":"2024-09-16T06:48:12.416727+0000","last_became_active":"2024-09-16T06:48:09.894853+0000","last_became_peered":"2024-09-16T06:48:09.894853+0000","last_unstale":"2024-09-16T06:48:12.416727+0000","last_undegraded":"2024-09-16T06:48:12.416727+0000","last_fullsized":"2024-09-16T06:48:12.416727+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:26.719521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.417890+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:48:12.417890+0000","last_peered":"2024-09-16T06:48:12.417890+0000","last_clean":"2024-09-16T06:48:12.417890+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:48:12.417890+0000","last_undegraded":"2024-09-16T06:48:12.417890+0000","last_fullsized":"2024-09-16T06:48:12.417890+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":17,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.768932+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:48:11.768932+0000","last_peered":"2024-09-16T06:48:11.768932+0000","last_clean":"2024-09-16T06:48:11.768932+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:48:11.768932+0000","last_undegraded":"2024-09-16T06:48:11.768932+0000","last_fullsized":"2024-09-16T06:48:11.768932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-16T06:48:16.801317+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:48:16.801317+0000","last_peered":"2024-09-16T06:48:16.801317+0000","last_clean":"2024-09-16T06:48:16.801317+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:48:16.801317+0000","last_undegraded":"2024-09-16T06:48:16.801317+0000","last_fullsized":"2024-09-16T06:48:16.801317+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:11.766789+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:48:11.766789+0000","last_peered":"2024-09-16T06:48:11.766789+0000","last_clean":"2024-09-16T06:48:11.766789+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:48:11.766789+0000","last_undegraded":"2024-09-16T06:48:11.766789+0000","last_fullsized":"2024-09-16T06:48:11.766789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:16.422076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"26'1","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-16T06:48:16.801307+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:48:16.801307+0000","last_peered":"2024-09-16T06:48:16.801307+0000","last_clean":"2024-09-16T06:48:16.801307+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:48:16.801307+0000","last_undegraded":"2024-09-16T06:48:16.801307+0000","last_fullsized":"2024-09-16T06:48:16.801307+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"28'157","reported_seq":209,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-16T06:48:12.027613+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:48:12.027613+0000","last_peered":"2024-09-16T06:48:12.027613+0000","last_clean":"2024-09-16T06:48:12.027613+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:48:12.027613+0000","last_undegraded":"2024-09-16T06:48:12.027613+0000","last_fullsized":"2024-09-16T06:48:12.027613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.138478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247812,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,6,7,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]}]},{"osd":14,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":127032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5249999999999999}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":117871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.984}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]}]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]}]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,5,6,7,8,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.542}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]}]},{"osd":5,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,4,6,7,8,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]}]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":827392,"data_stored":646780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]}]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,6,7,9,10,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4119999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4220000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3799999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3079999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.657}]}]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.502}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":117852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.619}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8360000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]}]},{"osd":9,"up_from":23,"seq":98784247812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":315392,"data_stored":127051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6910000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0190000000000001}]}]},{"osd":10,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":127032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,5,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:48:17.988 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [1, 2, 3, 12] 2024-09-16T06:48:17.988 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.3', '1', '2', '3', '12'] 2024-09-16T06:48:17.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.3 1 2 3 12 2024-09-16T06:48:18.054 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:18.053+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:18.076 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.076 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:48:18.156 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:18.151+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:18.240 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.240 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.253 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:48:18.256 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:18.257+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:18.357 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:18.357+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:18.358 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.358 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:48:18.459 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:18.457+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:18.485 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.485 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:48:18.559 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:18.561+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:18.612 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.612 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:48:18.632 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:18.660 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:18.660 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:18.660 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:18.661+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:18.730 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.731 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.743 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:48:18.761 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:18.761+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:18.858 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:18.858 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:18.862 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:18.861+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:18.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:48:18.882 INFO:teuthology.orchestra.run.smithi100.stderr:set 2.3 pg_upmap_items mapping to [1->2,3->12] 2024-09-16T06:48:18.962 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:18.959+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:19.001 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.002 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.015 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:48:19.063 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:19.065+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:19.133 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.134 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:48:19.190 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:19.193+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:19.265 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:19.265+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:19.310 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.310 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.324 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:48:19.366 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:19.369+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:19.466 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:19.463+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:19.486 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.486 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:48:19.567 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:19.569+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:19.622 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.622 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.636 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:48:19.667 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:19.669+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:19.769 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:19.772+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:19.799 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.799 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.814 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:48:19.857 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'unique_pool_0' created 2024-09-16T06:48:19.870 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:19.869+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:19.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:48:19.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-16T06:48:19.936 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:19.936 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:19.950 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:48:19.994 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:19.997+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:20.071 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:20.072 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:20.072 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:20.073+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:20.086 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:48:20.174 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:20.177+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:20.254 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:20.254 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:20.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:48:20.275 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:20.271+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:20.375 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:20.371+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:20.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:48:20.494 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:20.497+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:20.573 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:20.577 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:20.577+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:20.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:48:20.678 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:20.679 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:20.681+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:20.753 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:20.753 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:20.768 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:48:20.779 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:20.777+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:20.840 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:20.841 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:20.856 INFO:teuthology.orchestra.run.smithi100.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-16T06:48:20.856 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:48:20.872 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:20.872 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:20.880 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:20.881+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:20.900 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:20.911 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:20.911 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:20.911 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:20.911 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:20.924 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:20.925 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:20.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:48:20.944 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:20.944 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:20.970 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:20.980 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:20.980+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:20.997 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:21.032 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:21.046 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:21.046 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:21.047 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:21.047 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:21.047 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:21.057 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:21.058 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:21.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:48:21.081 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:21.080+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:21.088 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:21.088 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:21.115 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:21.115 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:21.150 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:21.150 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:21.176 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:21.177 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:21.181 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:21.181+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:21.185 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:21.185 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:21.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:21.185 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [] 2024-09-16T06:48:21.185 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:21.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:48:21.207 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:21.207 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:21.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:21.208+0000 7fd9cffff640 0 -- 172.21.15.100:0/1108629964 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7fd9d021e980 msgr2=0x7fd9d0220d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:21.282 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:21.280+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:21.320 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:21.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:21.320 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:21.320 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:21.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:48:21.384 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:21.381+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:21.456 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:21.457 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:21.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:48:21.517 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:21.515+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:21.586 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:21.588+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.942224399000001, 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.597 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:48:21.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:48:21.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.310572262999999, 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017661530000000002, 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282228+0000", 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282240+0000", 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2339e-05 2024-09-16T06:48:21.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282259+0000", 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8499e-05 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282274+0000", 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5673000000000001e-05 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297469+0000", 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015194177999999999 2024-09-16T06:48:21.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297683+0000", 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021437800000000001 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297867+0000", 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018445199999999999 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299716+0000", 2024-09-16T06:48:21.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018485590000000001 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299769+0000", 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3192999999999997e-05 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299836+0000", 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7255000000000006e-05 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299891+0000", 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4243999999999998e-05 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.310511583, 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018410340000000001, 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282289+0000", 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282302+0000", 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3049e-05 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282319+0000", 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7379000000000001e-05 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282336+0000", 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6881e-05 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.298971+0000", 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016634903999999999 2024-09-16T06:48:21.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299117+0000", 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014632600000000001 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299317+0000", 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019987600000000001 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300036+0000", 2024-09-16T06:48:21.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071873499999999997 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300588+0000", 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055181800000000004 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300655+0000", 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7742000000000003e-05 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300700+0000", 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999998e-05 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.128424221, 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:48:21.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:48:21.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.128295669, 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:48:21.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.128258342000001, 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:48:21.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:48:21.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.128123344, 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.128091183, 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:48:21.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:48:21.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.128062752, 2024-09-16T06:48:21.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:48:21.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:48:21.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.127955833, 2024-09-16T06:48:21.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:21.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:48:21.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:48:21.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.915942+0000", 2024-09-16T06:48:21.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.67685975, 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017381705000000001, 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 160 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915942+0000", 2024-09-16T06:48:21.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915942+0000", 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915941+0000", 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915950+0000", 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9389999999999992e-06 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915961+0000", 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0509000000000001e-05 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915968+0000", 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.17e-06 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.920119+0000", 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041510679999999999 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.920251+0000", 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013195899999999999 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.920423+0000", 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017157899999999999 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924441+0000", 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040179550000000001 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933252+0000", 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0088109409999999992 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933291+0000", 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9202000000000002e-05 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933323+0000", 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2481999999999998e-05 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.915976+0000", 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.676825131999999, 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017509480000000001, 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 161 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915976+0000", 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915976+0000", 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.915976+0000", 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916130+0000", 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015396999999999999 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916144+0000", 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3978999999999999e-05 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916155+0000", 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.920877+0000", 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004721289 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.920974+0000", 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7695000000000006e-05 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.921146+0000", 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017228299999999999 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927130+0000", 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059835189999999996 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933376+0000", 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062463279999999998 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933440+0000", 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3441000000000005e-05 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933486+0000", 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5986999999999999e-05 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.916165+0000", 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.676636651999999, 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017422329, 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 162 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916165+0000", 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916165+0000", 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916164+0000", 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916170+0000", 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2729999999999999e-06 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916180+0000", 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8160000000000005e-06 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916187+0000", 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7100000000000001e-06 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.921682+0000", 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054952760000000003 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.921820+0000", 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000137541 2024-09-16T06:48:21.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.922009+0000", 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000188958 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927134+0000", 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005125012 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933532+0000", 2024-09-16T06:48:21.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063986340000000003 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933561+0000", 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9232999999999999e-05 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933587+0000", 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5629000000000001e-05 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.916195+0000", 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.676606757, 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017564705, 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 163 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916195+0000", 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916195+0000", 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916194+0000", 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916200+0000", 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4870000000000002e-06 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916209+0000", 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1919999999999994e-06 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916216+0000", 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8229999999999997e-06 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.922447+0000", 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006231739 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.922510+0000", 2024-09-16T06:48:21.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2419e-05 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.922618+0000", 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107805 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927137+0000", 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045191169999999996 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933653+0000", 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065161079999999996 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933712+0000", 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9663000000000001e-05 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933759+0000", 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6959000000000002e-05 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.916223+0000", 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.676578143, 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017624425999999999, 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 164 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916223+0000", 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916223+0000", 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916223+0000", 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916348+0000", 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012531900000000001 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916355+0000", 2024-09-16T06:48:21.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7819999999999996e-06 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916359+0000", 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7699999999999999e-06 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.922950+0000", 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065917900000000001 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.923069+0000", 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118744 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.923254+0000", 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018496499999999999 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927140+0000", 2024-09-16T06:48:21.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038855280000000001 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933795+0000", 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066554450000000003 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933825+0000", 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9856e-05 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.933848+0000", 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2819999999999998e-05 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.676437854, 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018030338, 2024-09-16T06:48:21.689 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 165 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916363+0000", 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916367+0000", 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4790000000000001e-06 2024-09-16T06:48:21.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916371+0000", 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3460000000000004e-06 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916374+0000", 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4400000000000001e-06 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924029+0000", 2024-09-16T06:48:21.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076542229999999999 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924097+0000", 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8849000000000004e-05 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924193+0000", 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5544999999999994e-05 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927143+0000", 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029495250000000002 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934287+0000", 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071441270000000001 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934346+0000", 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9521000000000002e-05 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934394+0000", 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-05 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.434449322000001, 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:48:21.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:48:21.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:48:21.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.434419074000001, 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:48:21.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:48:21.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:48:21.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.434389639000001, 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:21.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:48:21.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.43427872, 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:21.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:21.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:48:21.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:21.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:21.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:21.719 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:21.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:48:21.721 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:21.721+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:21.821 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:21.820+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:21.886 DEBUG:teuthology.orchestra.run.smithi117:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-16T06:48:21.891 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:21.892 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:21.906 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight read -> 100 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight delete -> 50 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight snap_create -> 50 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight snap_remove -> 50 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight rollback -> 50 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight setattr -> 25 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight rmattr -> 25 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight copy_from -> 50 2024-09-16T06:48:21.909 INFO:tasks.rados.rados.0.smithi117.stdout:adding op weight append -> 100 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout:ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout:Configuration: 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout: Number of operations: 400 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout: Number of objects: 50 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout: Max in flight operations: 16 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout: Object size (in bytes): 4000000 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout: Write stride min: 400000 2024-09-16T06:48:21.910 INFO:tasks.rados.rados.0.smithi117.stdout: Write stride max: 800000 2024-09-16T06:48:21.922 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:21.924+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:22.023 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:22.020+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:22.028 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:22.028 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:22.042 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:48:22.124 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:22.123+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:22.158 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:22.158 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [] 2024-09-16T06:48:22.158 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:22.172 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:48:22.225 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:22.224+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:22.326 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:22.328+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:22.338 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:22.338 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:22.353 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:48:22.356 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:22.356 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:22.400 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:22.401 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:22.427 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:22.424+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:22.473 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:22.473 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:22.474 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:22.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:48:22.532 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:22.531+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4293.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.664256+0000", 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.946154988, 2024-09-16T06:48:22.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037402400000000001, 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4293", 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.100:0/1109988992", 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.664256+0000", 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.664256+0000", 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.664254+0000", 2024-09-16T06:48:22.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.664284+0000", 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9944999999999999e-05 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.664308+0000", 2024-09-16T06:48:22.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3995e-05 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.664323+0000", 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5682e-05 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.665167+0000", 2024-09-16T06:48:22.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084365199999999997 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.665492+0000", 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032528100000000002 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 15", 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.665805+0000", 2024-09-16T06:48:22.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031313800000000002 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667296+0000", 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014911119999999999 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667768+0000", 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047186 2024-09-16T06:48:22.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667894+0000", 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000125695 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667996+0000", 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102017 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:22.617 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4293.0:1 2.3 e26/25 ondisk, result = 0)", 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.667539+0000", 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.942871526999999, 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000471151, 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:22.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667539+0000", 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667539+0000", 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667537+0000", 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667549+0000", 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1596e-05 2024-09-16T06:48:22.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667622+0000", 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2731999999999994e-05 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667636+0000", 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4559000000000001e-05 2024-09-16T06:48:22.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667729+0000", 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2256000000000006e-05 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.667752+0000", 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3492000000000001e-05 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:22.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.668010+0000", 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025829599999999998 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:22.623 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:22.625 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:48:22.628 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:22.629+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:22.729 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:22.727+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:22.757 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:22.757 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:22.757 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:22.757 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:22.771 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:48:22.830 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:22.829+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:22.885 INFO:tasks.rados.rados.0.smithi117.stdout:1: write initial oid 1 2024-09-16T06:48:22.885 INFO:tasks.rados.rados.0.smithi117.stdout:1: seq_num 0 ranges {0=32768} 2024-09-16T06:48:22.885 INFO:tasks.rados.rados.0.smithi117.stdout:1: writing smithi11731613-1 from 0 to 32768 tid 1 2024-09-16T06:48:22.885 INFO:tasks.rados.rados.0.smithi117.stdout:2: write initial oid 2 2024-09-16T06:48:22.885 INFO:tasks.rados.rados.0.smithi117.stdout:2: seq_num 1 ranges {0=32768} 2024-09-16T06:48:22.886 INFO:tasks.rados.rados.0.smithi117.stdout:2: writing smithi11731613-2 from 0 to 32768 tid 1 2024-09-16T06:48:22.886 INFO:tasks.rados.rados.0.smithi117.stdout:3: write initial oid 3 2024-09-16T06:48:22.886 INFO:tasks.rados.rados.0.smithi117.stdout:3: seq_num 2 ranges {0=49152} 2024-09-16T06:48:22.886 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:3: writing smithi11731613-3 from 0 to 49152 tid 1 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:4: write initial oid 4 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:4: seq_num 3 ranges {0=65536} 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:4: writing smithi11731613-4 from 0 to 65536 tid 1 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:5: write initial oid 5 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:5: seq_num 4 ranges {0=81920} 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:5: writing smithi11731613-5 from 0 to 81920 tid 1 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:6: write initial oid 6 2024-09-16T06:48:22.888 INFO:tasks.rados.rados.0.smithi117.stdout:6: seq_num 5 ranges {0=98304} 2024-09-16T06:48:22.889 INFO:tasks.rados.rados.0.smithi117.stdout:6: writing smithi11731613-6 from 0 to 98304 tid 1 2024-09-16T06:48:22.889 INFO:tasks.rados.rados.0.smithi117.stdout:7: write initial oid 7 2024-09-16T06:48:22.889 INFO:tasks.rados.rados.0.smithi117.stdout:7: seq_num 6 ranges {0=114688} 2024-09-16T06:48:22.890 INFO:tasks.rados.rados.0.smithi117.stdout:7: writing smithi11731613-7 from 0 to 114688 tid 1 2024-09-16T06:48:22.890 INFO:tasks.rados.rados.0.smithi117.stdout:8: write initial oid 8 2024-09-16T06:48:22.890 INFO:tasks.rados.rados.0.smithi117.stdout:8: seq_num 7 ranges {0=131072} 2024-09-16T06:48:22.890 INFO:tasks.rados.rados.0.smithi117.stdout:8: writing smithi11731613-8 from 0 to 131072 tid 1 2024-09-16T06:48:22.890 INFO:tasks.rados.rados.0.smithi117.stdout:9: write initial oid 9 2024-09-16T06:48:22.890 INFO:tasks.rados.rados.0.smithi117.stdout:9: seq_num 8 ranges {0=147456} 2024-09-16T06:48:22.891 INFO:tasks.rados.rados.0.smithi117.stdout:9: writing smithi11731613-9 from 0 to 147456 tid 1 2024-09-16T06:48:22.891 INFO:tasks.rados.rados.0.smithi117.stdout:10: write initial oid 10 2024-09-16T06:48:22.892 INFO:tasks.rados.rados.0.smithi117.stdout:10: seq_num 9 ranges {0=163840} 2024-09-16T06:48:22.892 INFO:tasks.rados.rados.0.smithi117.stdout:10: writing smithi11731613-10 from 0 to 163840 tid 1 2024-09-16T06:48:22.893 INFO:tasks.rados.rados.0.smithi117.stdout:11: write initial oid 11 2024-09-16T06:48:22.893 INFO:tasks.rados.rados.0.smithi117.stdout:11: seq_num 10 ranges {0=180224} 2024-09-16T06:48:22.894 INFO:tasks.rados.rados.0.smithi117.stdout:11: writing smithi11731613-11 from 0 to 180224 tid 1 2024-09-16T06:48:22.894 INFO:tasks.rados.rados.0.smithi117.stdout:12: write initial oid 12 2024-09-16T06:48:22.894 INFO:tasks.rados.rados.0.smithi117.stdout:12: seq_num 11 ranges {0=196608} 2024-09-16T06:48:22.896 INFO:tasks.rados.rados.0.smithi117.stdout:12: writing smithi11731613-12 from 0 to 196608 tid 1 2024-09-16T06:48:22.896 INFO:tasks.rados.rados.0.smithi117.stdout:13: write initial oid 13 2024-09-16T06:48:22.896 INFO:tasks.rados.rados.0.smithi117.stdout:13: seq_num 12 ranges {0=212992} 2024-09-16T06:48:22.899 INFO:tasks.rados.rados.0.smithi117.stdout:13: writing smithi11731613-13 from 0 to 212992 tid 1 2024-09-16T06:48:22.899 INFO:tasks.rados.rados.0.smithi117.stdout:14: write initial oid 14 2024-09-16T06:48:22.899 INFO:tasks.rados.rados.0.smithi117.stdout:14: seq_num 13 ranges {0=229376} 2024-09-16T06:48:22.901 INFO:tasks.rados.rados.0.smithi117.stdout:14: writing smithi11731613-14 from 0 to 229376 tid 1 2024-09-16T06:48:22.901 INFO:tasks.rados.rados.0.smithi117.stdout:15: write initial oid 15 2024-09-16T06:48:22.901 INFO:tasks.rados.rados.0.smithi117.stdout:15: seq_num 14 ranges {0=245760} 2024-09-16T06:48:22.902 INFO:tasks.rados.rados.0.smithi117.stdout:15: writing smithi11731613-15 from 0 to 245760 tid 1 2024-09-16T06:48:22.903 INFO:tasks.rados.rados.0.smithi117.stdout:16: write initial oid 16 2024-09-16T06:48:22.903 INFO:tasks.rados.rados.0.smithi117.stdout:16: seq_num 15 ranges {0=262144} 2024-09-16T06:48:22.905 INFO:tasks.rados.rados.0.smithi117.stdout:16: writing smithi11731613-16 from 0 to 262144 tid 1 2024-09-16T06:48:22.905 INFO:tasks.rados.rados.0.smithi117.stdout:1: finishing write tid 1 to smithi11731613-1 2024-09-16T06:48:22.905 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:22.907 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:22.907 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:22.907 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:22.907 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:22.907 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:22.909 INFO:tasks.rados.rados.0.smithi117.stdout:2: finishing write tid 1 to smithi11731613-2 2024-09-16T06:48:22.913 INFO:tasks.rados.rados.0.smithi117.stdout:10: finishing write tid 1 to smithi11731613-10 2024-09-16T06:48:22.915 INFO:tasks.rados.rados.0.smithi117.stdout:1: finishing write tid 2 to smithi11731613-1 2024-09-16T06:48:22.916 INFO:tasks.rados.rados.0.smithi117.stdout:10: finishing write tid 2 to smithi11731613-10 2024-09-16T06:48:22.917 INFO:tasks.rados.rados.0.smithi117.stdout:4: finishing write tid 1 to smithi11731613-4 2024-09-16T06:48:22.917 INFO:tasks.rados.rados.0.smithi117.stdout:5: finishing write tid 1 to smithi11731613-5 2024-09-16T06:48:22.918 INFO:tasks.rados.rados.0.smithi117.stdout:4: finishing write tid 2 to smithi11731613-4 2024-09-16T06:48:22.918 INFO:tasks.rados.rados.0.smithi117.stdout:1: finishing write tid 3 to smithi11731613-1 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:1: oid 1 updating version 0 to 2 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:1: oid 1 version 2 is already newer than 1 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:1: done (15 left) 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:17: write initial oid 17 2024-09-16T06:48:22.919 INFO:tasks.rados.rados.0.smithi117.stdout:17: seq_num 16 ranges {0=278528} 2024-09-16T06:48:22.921 INFO:tasks.rados.rados.0.smithi117.stdout:17: writing smithi11731613-17 from 0 to 278528 tid 1 2024-09-16T06:48:22.921 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:22.921 INFO:tasks.rados.rados.0.smithi117.stdout:3: finishing write tid 1 to smithi11731613-3 2024-09-16T06:48:22.921 INFO:tasks.rados.rados.0.smithi117.stdout:6: finishing write tid 1 to smithi11731613-6 2024-09-16T06:48:22.921 INFO:tasks.rados.rados.0.smithi117.stdout:5: finishing write tid 2 to smithi11731613-5 2024-09-16T06:48:22.921 INFO:tasks.rados.rados.0.smithi117.stdout:2: finishing write tid 2 to smithi11731613-2 2024-09-16T06:48:22.922 INFO:tasks.rados.rados.0.smithi117.stdout:8: finishing write tid 1 to smithi11731613-8 2024-09-16T06:48:22.923 INFO:tasks.rados.rados.0.smithi117.stdout:8: finishing write tid 2 to smithi11731613-8 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:3: finishing write tid 2 to smithi11731613-3 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:10: finishing write tid 3 to smithi11731613-10 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:10: oid 10 updating version 0 to 1 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:10: oid 10 updating version 1 to 2 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 2 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:10: done (15 left) 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:18: write initial oid 18 2024-09-16T06:48:22.924 INFO:tasks.rados.rados.0.smithi117.stdout:18: seq_num 17 ranges {0=294912} 2024-09-16T06:48:22.926 INFO:tasks.rados.rados.0.smithi117.stdout:18: writing smithi11731613-18 from 0 to 294912 tid 1 2024-09-16T06:48:22.926 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:22.926 INFO:tasks.rados.rados.0.smithi117.stdout:7: finishing write tid 1 to smithi11731613-7 2024-09-16T06:48:22.926 INFO:tasks.rados.rados.0.smithi117.stdout:15: finishing write tid 1 to smithi11731613-15 2024-09-16T06:48:22.926 INFO:tasks.rados.rados.0.smithi117.stdout:4: finishing write tid 3 to smithi11731613-4 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:4: oid 4 updating version 0 to 1 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:4: oid 4 updating version 1 to 2 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:2: finishing write tid 3 to smithi11731613-2 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:2: oid 2 updating version 0 to 1 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:2: oid 2 updating version 1 to 2 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:2: done (15 left) 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:4: done (14 left) 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:19: write initial oid 19 2024-09-16T06:48:22.927 INFO:tasks.rados.rados.0.smithi117.stdout:19: seq_num 18 ranges {0=311296} 2024-09-16T06:48:22.931 INFO:tasks.rados.rados.0.smithi117.stdout:19: writing smithi11731613-19 from 0 to 311296 tid 1 2024-09-16T06:48:22.931 INFO:tasks.rados.rados.0.smithi117.stdout:20: write initial oid 20 2024-09-16T06:48:22.931 INFO:tasks.rados.rados.0.smithi117.stdout:20: seq_num 19 ranges {0=327680} 2024-09-16T06:48:22.932 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:20: writing smithi11731613-20 from 0 to 327680 tid 1 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:5: finishing write tid 3 to smithi11731613-5 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:5: oid 5 updating version 0 to 1 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:5: oid 5 updating version 1 to 2 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:11: finishing write tid 1 to smithi11731613-11 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:5: done (15 left) 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:21: write initial oid 21 2024-09-16T06:48:22.935 INFO:tasks.rados.rados.0.smithi117.stdout:21: seq_num 20 ranges {0=344064} 2024-09-16T06:48:22.936 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:22.936+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:21: writing smithi11731613-21 from 0 to 344064 tid 1 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:16: finishing write tid 1 to smithi11731613-16 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:7: finishing write tid 2 to smithi11731613-7 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:16: finishing write tid 2 to smithi11731613-16 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:17: finishing write tid 1 to smithi11731613-17 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:6: finishing write tid 2 to smithi11731613-6 2024-09-16T06:48:22.939 INFO:tasks.rados.rados.0.smithi117.stdout:12: finishing write tid 1 to smithi11731613-12 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:7: finishing write tid 3 to smithi11731613-7 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:7: oid 7 updating version 0 to 3 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:7: oid 7 updating version 3 to 4 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 7 v 4 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:12: finishing write tid 2 to smithi11731613-12 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:14: finishing write tid 1 to smithi11731613-14 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:14: finishing write tid 2 to smithi11731613-14 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:11: finishing write tid 2 to smithi11731613-11 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:9: finishing write tid 1 to smithi11731613-9 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:13: finishing write tid 1 to smithi11731613-13 2024-09-16T06:48:22.940 INFO:tasks.rados.rados.0.smithi117.stdout:7: done (15 left) 2024-09-16T06:48:22.941 INFO:tasks.rados.rados.0.smithi117.stdout:22: write initial oid 22 2024-09-16T06:48:22.941 INFO:tasks.rados.rados.0.smithi117.stdout:22: seq_num 21 ranges {0=360448} 2024-09-16T06:48:22.944 INFO:tasks.rados.rados.0.smithi117.stdout:22: writing smithi11731613-22 from 0 to 360448 tid 1 2024-09-16T06:48:22.944 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:9: finishing write tid 2 to smithi11731613-9 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:16: finishing write tid 3 to smithi11731613-16 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:16: oid 16 updating version 0 to 1 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:16: oid 16 updating version 1 to 2 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:3: finishing write tid 3 to smithi11731613-3 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:3: oid 3 updating version 0 to 2 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:3: oid 3 version 2 is already newer than 1 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-16T06:48:22.945 INFO:tasks.rados.rados.0.smithi117.stdout:9: finishing write tid 3 to smithi11731613-9 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:9: oid 9 updating version 0 to 6 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:9: oid 9 version 6 is already newer than 5 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:8: finishing write tid 3 to smithi11731613-8 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:8: oid 8 updating version 0 to 1 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:8: oid 8 updating version 1 to 2 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:14: finishing write tid 3 to smithi11731613-14 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:14: oid 14 updating version 0 to 1 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:14: oid 14 updating version 1 to 2 2024-09-16T06:48:22.946 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 14 v 2 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:6: finishing write tid 3 to smithi11731613-6 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:6: oid 6 updating version 0 to 1 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:6: oid 6 updating version 1 to 2 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:13: finishing write tid 2 to smithi11731613-13 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:12: finishing write tid 3 to smithi11731613-12 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:12: oid 12 updating version 0 to 3 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:12: oid 12 updating version 3 to 4 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 12 v 4 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:13: finishing write tid 3 to smithi11731613-13 2024-09-16T06:48:22.947 INFO:tasks.rados.rados.0.smithi117.stdout:13: oid 13 updating version 0 to 4 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:13: oid 13 version 4 is already newer than 3 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:11: finishing write tid 3 to smithi11731613-11 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:11: oid 11 updating version 0 to 3 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:11: oid 11 updating version 3 to 4 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:15: finishing write tid 2 to smithi11731613-15 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:18: finishing write tid 1 to smithi11731613-18 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:3: done (15 left) 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:6: done (14 left) 2024-09-16T06:48:22.948 INFO:tasks.rados.rados.0.smithi117.stdout:8: done (13 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:9: done (12 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:11: done (11 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:12: done (10 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:13: done (9 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:14: done (8 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:16: done (7 left) 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:23: write initial oid 23 2024-09-16T06:48:22.949 INFO:tasks.rados.rados.0.smithi117.stdout:23: seq_num 22 ranges {0=376832} 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:23: writing smithi11731613-23 from 0 to 376832 tid 1 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:24: write initial oid 24 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:20: finishing write tid 1 to smithi11731613-20 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:19: finishing write tid 1 to smithi11731613-19 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:21: finishing write tid 1 to smithi11731613-21 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:19: finishing write tid 2 to smithi11731613-19 2024-09-16T06:48:22.950 INFO:tasks.rados.rados.0.smithi117.stdout:24: seq_num 23 ranges {0=393216} 2024-09-16T06:48:22.954 INFO:tasks.rados.rados.0.smithi117.stdout:24: writing smithi11731613-24 from 0 to 393216 tid 1 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:21: finishing write tid 2 to smithi11731613-21 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:15: finishing write tid 3 to smithi11731613-15 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:15: oid 15 updating version 0 to 3 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:15: oid 15 updating version 3 to 4 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 15 v 4 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-16T06:48:22.955 INFO:tasks.rados.rados.0.smithi117.stdout:21: finishing write tid 3 to smithi11731613-21 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:21: oid 21 updating version 0 to 7 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:21: oid 21 updating version 7 to 8 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 21 v 8 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:17: finishing write tid 2 to smithi11731613-17 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:22: finishing write tid 1 to smithi11731613-22 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:22: finishing write tid 2 to smithi11731613-22 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:15: done (8 left) 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:21: done (7 left) 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:25: write initial oid 25 2024-09-16T06:48:22.956 INFO:tasks.rados.rados.0.smithi117.stdout:25: seq_num 24 ranges {0=409600} 2024-09-16T06:48:22.959 INFO:tasks.rados.rados.0.smithi117.stdout:25: writing smithi11731613-25 from 0 to 409600 tid 1 2024-09-16T06:48:22.959 INFO:tasks.rados.rados.0.smithi117.stdout:26: write initial oid 26 2024-09-16T06:48:22.959 INFO:tasks.rados.rados.0.smithi117.stdout:26: seq_num 25 ranges {0=425984} 2024-09-16T06:48:22.964 INFO:tasks.rados.rados.0.smithi117.stdout:26: writing smithi11731613-26 from 0 to 425984 tid 1 2024-09-16T06:48:22.964 INFO:tasks.rados.rados.0.smithi117.stdout:22: finishing write tid 3 to smithi11731613-22 2024-09-16T06:48:22.964 INFO:tasks.rados.rados.0.smithi117.stdout:22: oid 22 updating version 0 to 4 2024-09-16T06:48:22.964 INFO:tasks.rados.rados.0.smithi117.stdout:22: oid 22 version 4 is already newer than 3 2024-09-16T06:48:22.964 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:19: finishing write tid 3 to smithi11731613-19 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:19: oid 19 updating version 0 to 5 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:19: oid 19 updating version 5 to 6 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:23: finishing write tid 1 to smithi11731613-23 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:24: finishing write tid 1 to smithi11731613-24 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:24: finishing write tid 2 to smithi11731613-24 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:25: finishing write tid 1 to smithi11731613-25 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:19: done (8 left) 2024-09-16T06:48:22.965 INFO:tasks.rados.rados.0.smithi117.stdout:22: done (7 left) 2024-09-16T06:48:22.966 INFO:tasks.rados.rados.0.smithi117.stdout:27: write initial oid 27 2024-09-16T06:48:22.966 INFO:tasks.rados.rados.0.smithi117.stdout:27: seq_num 26 ranges {0=442368} 2024-09-16T06:48:22.968 INFO:tasks.rados.rados.0.smithi117.stdout:27: writing smithi11731613-27 from 0 to 442368 tid 1 2024-09-16T06:48:22.968 INFO:tasks.rados.rados.0.smithi117.stdout:28: write initial oid 28 2024-09-16T06:48:22.968 INFO:tasks.rados.rados.0.smithi117.stdout:28: seq_num 27 ranges {0=458752} 2024-09-16T06:48:22.972 INFO:tasks.rados.rados.0.smithi117.stdout:28: writing smithi11731613-28 from 0 to 458752 tid 1 2024-09-16T06:48:22.972 INFO:tasks.rados.rados.0.smithi117.stdout:29: write initial oid 29 2024-09-16T06:48:22.972 INFO:tasks.rados.rados.0.smithi117.stdout:29: seq_num 28 ranges {0=475136} 2024-09-16T06:48:22.977 INFO:tasks.rados.rados.0.smithi117.stdout:29: writing smithi11731613-29 from 0 to 475136 tid 1 2024-09-16T06:48:22.977 INFO:tasks.rados.rados.0.smithi117.stdout:30: write initial oid 30 2024-09-16T06:48:22.977 INFO:tasks.rados.rados.0.smithi117.stdout:30: seq_num 29 ranges {0=491520} 2024-09-16T06:48:22.980 INFO:tasks.rados.rados.0.smithi117.stdout:30: writing smithi11731613-30 from 0 to 491520 tid 1 2024-09-16T06:48:22.980 INFO:tasks.rados.rados.0.smithi117.stdout:31: write initial oid 31 2024-09-16T06:48:22.980 INFO:tasks.rados.rados.0.smithi117.stdout:31: seq_num 30 ranges {0=507904} 2024-09-16T06:48:22.985 INFO:tasks.rados.rados.0.smithi117.stdout:31: writing smithi11731613-31 from 0 to 507904 tid 1 2024-09-16T06:48:22.985 INFO:tasks.rados.rados.0.smithi117.stdout:24: finishing write tid 3 to smithi11731613-24 2024-09-16T06:48:22.985 INFO:tasks.rados.rados.0.smithi117.stdout:24: oid 24 updating version 0 to 5 2024-09-16T06:48:22.985 INFO:tasks.rados.rados.0.smithi117.stdout:24: oid 24 updating version 5 to 6 2024-09-16T06:48:22.985 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-16T06:48:22.985 INFO:tasks.rados.rados.0.smithi117.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:27: finishing write tid 1 to smithi11731613-27 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:27: finishing write tid 2 to smithi11731613-27 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:28: finishing write tid 1 to smithi11731613-28 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:28: finishing write tid 2 to smithi11731613-28 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:27: finishing write tid 3 to smithi11731613-27 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:27: oid 27 updating version 0 to 8 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:27: oid 27 version 8 is already newer than 7 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 27 v 8 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:28: finishing write tid 3 to smithi11731613-28 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:28: oid 28 updating version 0 to 6 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:28: oid 28 version 6 is already newer than 5 2024-09-16T06:48:22.986 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:26: finishing write tid 1 to smithi11731613-26 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:24: done (11 left) 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:27: done (10 left) 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:28: done (9 left) 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:32: write initial oid 32 2024-09-16T06:48:22.987 INFO:tasks.rados.rados.0.smithi117.stdout:32: seq_num 31 ranges {0=524288} 2024-09-16T06:48:22.989 INFO:tasks.rados.rados.0.smithi117.stdout:32: writing smithi11731613-32 from 0 to 524288 tid 1 2024-09-16T06:48:22.989 INFO:tasks.rados.rados.0.smithi117.stdout:25: finishing write tid 2 to smithi11731613-25 2024-09-16T06:48:22.989 INFO:tasks.rados.rados.0.smithi117.stdout:30: finishing write tid 1 to smithi11731613-30 2024-09-16T06:48:22.989 INFO:tasks.rados.rados.0.smithi117.stdout:29: finishing write tid 1 to smithi11731613-29 2024-09-16T06:48:22.989 INFO:tasks.rados.rados.0.smithi117.stdout:29: finishing write tid 2 to smithi11731613-29 2024-09-16T06:48:22.990 INFO:tasks.rados.rados.0.smithi117.stdout:30: finishing write tid 2 to smithi11731613-30 2024-09-16T06:48:22.990 INFO:tasks.rados.rados.0.smithi117.stdout:33: write initial oid 33 2024-09-16T06:48:22.990 INFO:tasks.rados.rados.0.smithi117.stdout:33: seq_num 32 ranges {0=540672} 2024-09-16T06:48:22.993 INFO:tasks.rados.rados.0.smithi117.stdout:33: writing smithi11731613-33 from 0 to 540672 tid 1 2024-09-16T06:48:22.993 INFO:tasks.rados.rados.0.smithi117.stdout:34: write initial oid 34 2024-09-16T06:48:22.993 INFO:tasks.rados.rados.0.smithi117.stdout:34: seq_num 33 ranges {0=557056} 2024-09-16T06:48:22.997 INFO:tasks.rados.rados.0.smithi117.stdout:34: writing smithi11731613-34 from 0 to 557056 tid 1 2024-09-16T06:48:22.997 INFO:tasks.rados.rados.0.smithi117.stdout:35: write initial oid 35 2024-09-16T06:48:22.997 INFO:tasks.rados.rados.0.smithi117.stdout:35: seq_num 34 ranges {0=573440} 2024-09-16T06:48:23.000 INFO:tasks.rados.rados.0.smithi117.stdout:35: writing smithi11731613-35 from 0 to 573440 tid 1 2024-09-16T06:48:23.001 INFO:tasks.rados.rados.0.smithi117.stdout:36: write initial oid 36 2024-09-16T06:48:23.001 INFO:tasks.rados.rados.0.smithi117.stdout:36: seq_num 35 ranges {0=589824} 2024-09-16T06:48:23.004 INFO:tasks.rados.rados.0.smithi117.stdout:36: writing smithi11731613-36 from 0 to 589824 tid 1 2024-09-16T06:48:23.004 INFO:tasks.rados.rados.0.smithi117.stdout:37: write initial oid 37 2024-09-16T06:48:23.004 INFO:tasks.rados.rados.0.smithi117.stdout:37: seq_num 36 ranges {0=606208} 2024-09-16T06:48:23.008 INFO:tasks.rados.rados.0.smithi117.stdout:37: writing smithi11731613-37 from 0 to 606208 tid 1 2024-09-16T06:48:23.008 INFO:tasks.rados.rados.0.smithi117.stdout:38: write initial oid 38 2024-09-16T06:48:23.008 INFO:tasks.rados.rados.0.smithi117.stdout:38: seq_num 37 ranges {0=622592} 2024-09-16T06:48:23.012 INFO:tasks.rados.rados.0.smithi117.stdout:38: writing smithi11731613-38 from 0 to 622592 tid 1 2024-09-16T06:48:23.012 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:29: finishing write tid 3 to smithi11731613-29 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:29: oid 29 updating version 0 to 2 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:29: oid 29 version 2 is already newer than 1 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 29 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:25: finishing write tid 3 to smithi11731613-25 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:25: oid 25 updating version 0 to 5 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:25: oid 25 updating version 5 to 6 2024-09-16T06:48:23.013 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:31: finishing write tid 1 to smithi11731613-31 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:30: finishing write tid 3 to smithi11731613-30 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:30: oid 30 updating version 0 to 8 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:30: oid 30 version 8 is already newer than 7 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 8 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:32: finishing write tid 1 to smithi11731613-32 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:31: finishing write tid 2 to smithi11731613-31 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:23: finishing write tid 2 to smithi11731613-23 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:23: finishing write tid 3 to smithi11731613-23 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:23: oid 23 updating version 0 to 8 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:23: oid 23 version 8 is already newer than 7 2024-09-16T06:48:23.014 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 23 v 8 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:33: finishing write tid 1 to smithi11731613-33 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:23: done (15 left) 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:25: done (14 left) 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:29: done (13 left) 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:30: done (12 left) 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:39: write initial oid 39 2024-09-16T06:48:23.015 INFO:tasks.rados.rados.0.smithi117.stdout:39: seq_num 38 ranges {0=638976} 2024-09-16T06:48:23.016 INFO:tasks.rados.rados.0.smithi117.stdout:39: writing smithi11731613-39 from 0 to 638976 tid 1 2024-09-16T06:48:23.016 INFO:tasks.rados.rados.0.smithi117.stdout:40: write initial oid 40 2024-09-16T06:48:23.016 INFO:tasks.rados.rados.0.smithi117.stdout:40: seq_num 39 ranges {0=671744} 2024-09-16T06:48:23.020 INFO:tasks.rados.rados.0.smithi117.stdout:40: writing smithi11731613-40 from 0 to 671744 tid 1 2024-09-16T06:48:23.020 INFO:tasks.rados.rados.0.smithi117.stdout:41: write initial oid 41 2024-09-16T06:48:23.020 INFO:tasks.rados.rados.0.smithi117.stdout:41: seq_num 40 ranges {0=688128} 2024-09-16T06:48:23.024 INFO:tasks.rados.rados.0.smithi117.stdout:41: writing smithi11731613-41 from 0 to 688128 tid 1 2024-09-16T06:48:23.024 INFO:tasks.rados.rados.0.smithi117.stdout:42: write initial oid 42 2024-09-16T06:48:23.025 INFO:tasks.rados.rados.0.smithi117.stdout:42: seq_num 41 ranges {0=704512} 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout:42: writing smithi11731613-42 from 0 to 704512 tid 1 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout:20: finishing write tid 2 to smithi11731613-20 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout:34: finishing write tid 1 to smithi11731613-34 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout:34: finishing write tid 2 to smithi11731613-34 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout:35: finishing write tid 1 to smithi11731613-35 2024-09-16T06:48:23.030 INFO:tasks.rados.rados.0.smithi117.stdout:35: finishing write tid 2 to smithi11731613-35 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:37: finishing write tid 1 to smithi11731613-37 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:37: finishing write tid 2 to smithi11731613-37 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:36: finishing write tid 1 to smithi11731613-36 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:36: finishing write tid 2 to smithi11731613-36 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:35: finishing write tid 3 to smithi11731613-35 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:35: oid 35 updating version 0 to 1 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:35: oid 35 updating version 1 to 2 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 35 v 2 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-16T06:48:23.031 INFO:tasks.rados.rados.0.smithi117.stdout:39: finishing write tid 1 to smithi11731613-39 2024-09-16T06:48:23.032 INFO:tasks.rados.rados.0.smithi117.stdout:35: done (15 left) 2024-09-16T06:48:23.032 INFO:tasks.rados.rados.0.smithi117.stdout:43: write initial oid 43 2024-09-16T06:48:23.032 INFO:tasks.rados.rados.0.smithi117.stdout:43: seq_num 42 ranges {0=720896} 2024-09-16T06:48:23.033 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:23.032+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:23.035 INFO:tasks.rados.rados.0.smithi117.stdout:43: writing smithi11731613-43 from 0 to 720896 tid 1 2024-09-16T06:48:23.035 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:26: finishing write tid 2 to smithi11731613-26 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:36: finishing write tid 3 to smithi11731613-36 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:36: oid 36 updating version 0 to 3 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:36: oid 36 updating version 3 to 4 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 36 v 4 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:39: finishing write tid 2 to smithi11731613-39 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:37: finishing write tid 3 to smithi11731613-37 2024-09-16T06:48:23.036 INFO:tasks.rados.rados.0.smithi117.stdout:37: oid 37 updating version 0 to 4 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:37: oid 37 version 4 is already newer than 3 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 4 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:38: finishing write tid 1 to smithi11731613-38 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:40: finishing write tid 1 to smithi11731613-40 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:39: finishing write tid 3 to smithi11731613-39 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:39: oid 39 updating version 0 to 9 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:39: oid 39 updating version 9 to 10 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 39 v 10 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:26: finishing write tid 3 to smithi11731613-26 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:26: oid 26 updating version 0 to 1 2024-09-16T06:48:23.037 INFO:tasks.rados.rados.0.smithi117.stdout:26: oid 26 updating version 1 to 2 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 26 v 2 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:40: finishing write tid 2 to smithi11731613-40 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:38: finishing write tid 2 to smithi11731613-38 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:32: finishing write tid 2 to smithi11731613-32 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:41: finishing write tid 1 to smithi11731613-41 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:41: finishing write tid 2 to smithi11731613-41 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:40: finishing write tid 3 to smithi11731613-40 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:40: oid 40 updating version 0 to 3 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:40: oid 40 updating version 3 to 4 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 40 v 4 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:26: done (15 left) 2024-09-16T06:48:23.038 INFO:tasks.rados.rados.0.smithi117.stdout:36: done (14 left) 2024-09-16T06:48:23.039 INFO:tasks.rados.rados.0.smithi117.stdout:37: done (13 left) 2024-09-16T06:48:23.039 INFO:tasks.rados.rados.0.smithi117.stdout:39: done (12 left) 2024-09-16T06:48:23.039 INFO:tasks.rados.rados.0.smithi117.stdout:40: done (11 left) 2024-09-16T06:48:23.039 INFO:tasks.rados.rados.0.smithi117.stdout:44: write initial oid 44 2024-09-16T06:48:23.039 INFO:tasks.rados.rados.0.smithi117.stdout:44: seq_num 43 ranges {0=737280} 2024-09-16T06:48:23.039 INFO:tasks.rados.rados.0.smithi117.stdout:44: writing smithi11731613-44 from 0 to 737280 tid 1 2024-09-16T06:48:23.040 INFO:tasks.rados.rados.0.smithi117.stdout:45: write initial oid 45 2024-09-16T06:48:23.040 INFO:tasks.rados.rados.0.smithi117.stdout:45: seq_num 44 ranges {0=753664} 2024-09-16T06:48:23.044 INFO:tasks.rados.rados.0.smithi117.stdout:45: writing smithi11731613-45 from 0 to 753664 tid 1 2024-09-16T06:48:23.044 INFO:tasks.rados.rados.0.smithi117.stdout:46: write initial oid 46 2024-09-16T06:48:23.044 INFO:tasks.rados.rados.0.smithi117.stdout:46: seq_num 45 ranges {0=770048} 2024-09-16T06:48:23.049 INFO:tasks.rados.rados.0.smithi117.stdout:46: writing smithi11731613-46 from 0 to 770048 tid 1 2024-09-16T06:48:23.049 INFO:tasks.rados.rados.0.smithi117.stdout:47: write initial oid 47 2024-09-16T06:48:23.050 INFO:tasks.rados.rados.0.smithi117.stdout:47: seq_num 46 ranges {0=786432} 2024-09-16T06:48:23.055 INFO:tasks.rados.rados.0.smithi117.stdout:47: writing smithi11731613-47 from 0 to 786432 tid 1 2024-09-16T06:48:23.055 INFO:tasks.rados.rados.0.smithi117.stdout:48: write initial oid 48 2024-09-16T06:48:23.055 INFO:tasks.rados.rados.0.smithi117.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-16T06:48:23.057 INFO:tasks.rados.rados.0.smithi117.stdout:48: writing smithi11731613-48 from 0 to 425984 tid 1 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:48: writing smithi11731613-48 from 425984 to 802816 tid 2 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:41: finishing write tid 3 to smithi11731613-41 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:41: oid 41 updating version 0 to 9 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:41: oid 41 updating version 9 to 10 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 41 v 10 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-16T06:48:23.061 INFO:tasks.rados.rados.0.smithi117.stdout:41: done (15 left) 2024-09-16T06:48:23.062 INFO:tasks.rados.rados.0.smithi117.stdout:49: write initial oid 49 2024-09-16T06:48:23.062 INFO:tasks.rados.rados.0.smithi117.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-16T06:48:23.063 INFO:tasks.rados.rados.0.smithi117.stdout:49: writing smithi11731613-49 from 0 to 442368 tid 1 2024-09-16T06:48:23.066 INFO:tasks.rados.rados.0.smithi117.stdout:49: writing smithi11731613-49 from 442368 to 819200 tid 2 2024-09-16T06:48:23.066 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 16 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:38: finishing write tid 3 to smithi11731613-38 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:38: oid 38 updating version 0 to 4 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:38: oid 38 version 4 is already newer than 3 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 38 v 4 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:17: finishing write tid 3 to smithi11731613-17 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:17: oid 17 updating version 0 to 4 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:17: oid 17 version 4 is already newer than 3 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:20: finishing write tid 3 to smithi11731613-20 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:20: oid 20 updating version 0 to 4 2024-09-16T06:48:23.067 INFO:tasks.rados.rados.0.smithi117.stdout:20: oid 20 version 4 is already newer than 3 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 20 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:42: finishing write tid 1 to smithi11731613-42 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:34: finishing write tid 3 to smithi11731613-34 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:34: oid 34 updating version 0 to 2 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:34: oid 34 version 2 is already newer than 1 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 34 v 2 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:42: finishing write tid 2 to smithi11731613-42 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:44: finishing write tid 1 to smithi11731613-44 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:17: done (15 left) 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:20: done (14 left) 2024-09-16T06:48:23.068 INFO:tasks.rados.rados.0.smithi117.stdout:34: done (13 left) 2024-09-16T06:48:23.069 INFO:tasks.rados.rados.0.smithi117.stdout:38: done (12 left) 2024-09-16T06:48:23.069 INFO:tasks.rados.rados.0.smithi117.stdout:50: write initial oid 50 2024-09-16T06:48:23.069 INFO:tasks.rados.rados.0.smithi117.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-16T06:48:23.069 INFO:tasks.rados.rados.0.smithi117.stdout:50: writing smithi11731613-50 from 0 to 458752 tid 1 2024-09-16T06:48:23.071 INFO:tasks.rados.rados.0.smithi117.stdout:50: writing smithi11731613-50 from 458752 to 835584 tid 2 2024-09-16T06:48:23.071 INFO:tasks.rados.rados.0.smithi117.stdout:51: delete oid 20 current snap is 0 2024-09-16T06:48:23.071 INFO:tasks.rados.rados.0.smithi117.stdout:44: finishing write tid 2 to smithi11731613-44 2024-09-16T06:48:23.071 INFO:tasks.rados.rados.0.smithi117.stdout:32: finishing write tid 3 to smithi11731613-32 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:32: oid 32 updating version 0 to 10 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:32: oid 32 version 10 is already newer than 9 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 32 v 10 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:44: finishing write tid 3 to smithi11731613-44 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:44: oid 44 updating version 0 to 2 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:44: oid 44 version 2 is already newer than 1 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 2 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:31: finishing write tid 3 to smithi11731613-31 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:31: oid 31 updating version 0 to 7 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:31: oid 31 updating version 7 to 8 2024-09-16T06:48:23.072 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 31 v 8 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:45: finishing write tid 1 to smithi11731613-45 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:46: finishing write tid 1 to smithi11731613-46 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:45: finishing write tid 2 to smithi11731613-45 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:46: finishing write tid 2 to smithi11731613-46 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:47: finishing write tid 1 to smithi11731613-47 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:42: finishing write tid 3 to smithi11731613-42 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:42: oid 42 updating version 0 to 5 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:42: oid 42 updating version 5 to 6 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:46: finishing write tid 3 to smithi11731613-46 2024-09-16T06:48:23.073 INFO:tasks.rados.rados.0.smithi117.stdout:46: oid 46 updating version 0 to 4 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:46: oid 46 version 4 is already newer than 3 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 46 v 4 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:47: finishing write tid 2 to smithi11731613-47 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:48: finishing write tid 1 to smithi11731613-48 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:48: finishing write tid 2 to smithi11731613-48 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:47: finishing write tid 3 to smithi11731613-47 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:47: oid 47 updating version 0 to 6 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:47: oid 47 version 6 is already newer than 5 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 47 v 6 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-16T06:48:23.074 INFO:tasks.rados.rados.0.smithi117.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-16T06:48:23.075 INFO:tasks.rados.rados.0.smithi117.stdout:49: finishing write tid 1 to smithi11731613-49 2024-09-16T06:48:23.075 INFO:tasks.rados.rados.0.smithi117.stdout:43: finishing write tid 1 to smithi11731613-43 2024-09-16T06:48:23.077 INFO:tasks.rados.rados.0.smithi117.stdout:43: finishing write tid 2 to smithi11731613-43 2024-09-16T06:48:23.078 INFO:tasks.rados.rados.0.smithi117.stdout:31: done (13 left) 2024-09-16T06:48:23.078 INFO:tasks.rados.rados.0.smithi117.stdout:32: done (12 left) 2024-09-16T06:48:23.078 INFO:tasks.rados.rados.0.smithi117.stdout:42: done (11 left) 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:44: done (10 left) 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:46: done (9 left) 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:47: done (8 left) 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:51: done (7 left) 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:52: read oid 16 snap -1 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:52: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:53: delete oid 11 current snap is 0 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:49: finishing write tid 2 to smithi11731613-49 2024-09-16T06:48:23.079 INFO:tasks.rados.rados.0.smithi117.stdout:49: finishing write tid 3 to smithi11731613-49 2024-09-16T06:48:23.080 INFO:tasks.rados.rados.0.smithi117.stdout:50: finishing write tid 1 to smithi11731613-50 2024-09-16T06:48:23.080 INFO:tasks.rados.rados.0.smithi117.stdout:50: finishing write tid 2 to smithi11731613-50 2024-09-16T06:48:23.081 INFO:tasks.rados.rados.0.smithi117.stdout:33: finishing write tid 2 to smithi11731613-33 2024-09-16T06:48:23.081 INFO:tasks.rados.rados.0.smithi117.stdout:49: finishing write tid 4 to smithi11731613-49 2024-09-16T06:48:23.081 INFO:tasks.rados.rados.0.smithi117.stdout:49: oid 49 updating version 0 to 7 2024-09-16T06:48:23.081 INFO:tasks.rados.rados.0.smithi117.stdout:49: oid 49 version 7 is already newer than 5 2024-09-16T06:48:23.081 INFO:tasks.rados.rados.0.smithi117.stdout:49: oid 49 version 7 is already newer than 6 2024-09-16T06:48:23.081 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 49 v 7 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-16T06:48:23.082 INFO:tasks.rados.rados.0.smithi117.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-16T06:48:23.082 INFO:tasks.rados.rados.0.smithi117.stdout:43: finishing write tid 3 to smithi11731613-43 2024-09-16T06:48:23.082 INFO:tasks.rados.rados.0.smithi117.stdout:43: oid 43 updating version 0 to 5 2024-09-16T06:48:23.082 INFO:tasks.rados.rados.0.smithi117.stdout:43: oid 43 updating version 5 to 6 2024-09-16T06:48:23.082 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 43 v 6 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-16T06:48:23.082 INFO:tasks.rados.rados.0.smithi117.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-16T06:48:23.083 INFO:tasks.rados.rados.0.smithi117.stdout:33: finishing write tid 3 to smithi11731613-33 2024-09-16T06:48:23.083 INFO:tasks.rados.rados.0.smithi117.stdout:33: oid 33 updating version 0 to 9 2024-09-16T06:48:23.083 INFO:tasks.rados.rados.0.smithi117.stdout:33: oid 33 updating version 9 to 10 2024-09-16T06:48:23.083 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 33 v 10 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-16T06:48:23.083 INFO:tasks.rados.rados.0.smithi117.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4293.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:11.758972+0000", 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.327593723, 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011726535999999999, 2024-09-16T06:48:23.086 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4293", 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.100:0/1109988992", 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 7 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.758972+0000", 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.087 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.758972+0000", 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.758970+0000", 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.758986+0000", 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6436999999999999e-05 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.088 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.759022+0000", 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5599999999999998e-05 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.759041+0000", 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9046e-05 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.759150+0000", 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109446 2024-09-16T06:48:23.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.759162+0000", 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1569999999999999e-05 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.767030+0000", 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0078687689999999994 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.090 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.767270+0000", 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023910500000000001 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for subops from 9", 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.767754+0000", 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00048423200000000002 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op_commit", 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.769315+0000", 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001561642 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.091 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.770445+0000", 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011299750000000001 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.770577+0000", 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000131106 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:11.770699+0000", 2024-09-16T06:48:23.092 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012250499999999999 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds1 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.18851416600000001, 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018662743999999998, 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.093 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898050+0000", 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898094+0000", 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3769000000000001e-05 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915572+0000", 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017478235000000002 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915584+0000", 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1996e-05 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915648+0000", 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3551000000000002e-05 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915661+0000", 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2897e-05 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.096 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916697+0000", 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001036653 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916715+0000", 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7376000000000001e-05 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:28 3.as0 3:5d93488c:::smithi11731613-10:head [append 0~163840,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.905910+0000", 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.18065630599999999, 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0099862900000000001, 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 28 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.905910+0000", 2024-09-16T06:48:23.098 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.905910+0000", 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.905908+0000", 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906486+0000", 2024-09-16T06:48:23.099 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00057813700000000005 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906505+0000", 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8303000000000001e-05 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906516+0000", 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1694e-05 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.100 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906892+0000", 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037544800000000001 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.907213+0000", 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032158300000000001 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909261+0000", 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002047021 2024-09-16T06:48:23.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.912600+0000", 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0033398260000000002 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915829+0000", 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0032283749999999999 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915896+0000", 2024-09-16T06:48:23.102 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7502999999999994e-05 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:29 3.as0 3:5d93488c:::smithi11731613-10:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.906538+0000", 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.18002786900000001, 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011744533, 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.103 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 29 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906538+0000", 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906538+0000", 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.104 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906537+0000", 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906545+0000", 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2609999999999996e-06 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906556+0000", 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1005999999999999e-05 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.105 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906564+0000", 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6959999999999994e-06 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910538+0000", 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0039741239999999999 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910661+0000", 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012309300000000001 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.106 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911102+0000", 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00044026200000000002 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.912697+0000", 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001595108 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918230+0000", 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0055327520000000002 2024-09-16T06:48:23.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918283+0000", 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3338999999999999e-05 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:30 3.as0 3:5d93488c:::smithi11731613-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.17999472499999999, 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017066839, 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.108 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 30 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.109 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906575+0000", 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1219999999999997e-06 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.110 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906585+0000", 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0142e-05 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906593+0000", 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5340000000000002e-06 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911387+0000", 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0047939760000000001 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.111 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911456+0000", 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9554000000000005e-05 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918704+0000", 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072483469999999996 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918831+0000", 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012617499999999999 2024-09-16T06:48:23.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.923638+0000", 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048076789999999996 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds1 35/32 ECSubWrite(tid=5, reqid=client.4511.0:50, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.157747358, 2024-09-16T06:48:23.113 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023812890999999999, 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.114 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928818+0000", 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928835+0000", 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7011999999999999e-05 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949938+0000", 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.02110269 2024-09-16T06:48:23.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949947+0000", 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0010000000000007e-06 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950024+0000", 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6520000000000006e-05 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950032+0000", 2024-09-16T06:48:23.116 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6750000000000008e-06 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952592+0000", 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002559296 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952632+0000", 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0243000000000001e-05 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.117 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:64 3.as0 3:5ad2eba8:::smithi11731613-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.946349+0000", 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.14021763200000001, 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084651929999999993, 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.118 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 64 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946349+0000", 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946349+0000", 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.119 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946346+0000", 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946653+0000", 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030686 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946759+0000", 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105955 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.120 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946772+0000", 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3176999999999999e-05 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946844+0000", 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2125000000000005e-05 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.947009+0000", 2024-09-16T06:48:23.121 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016499900000000001 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.947795+0000", 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00078569600000000001 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952205+0000", 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0044104629999999999 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.122 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.954750+0000", 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002544897 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.954814+0000", 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3736000000000001e-05 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:65 3.as0 3:5ad2eba8:::smithi11731613-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.123 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.946790+0000", 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.13977587499999999, 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082461470000000005, 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 65 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946790+0000", 2024-09-16T06:48:23.124 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946790+0000", 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946790+0000", 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.125 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946795+0000", 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6359999999999999e-06 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946805+0000", 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7769999999999997e-06 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946815+0000", 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0022e-05 2024-09-16T06:48:23.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948600+0000", 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001785401 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948709+0000", 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109064 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.127 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949421+0000", 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00071128000000000001 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952725+0000", 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0033045739999999998 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.128 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.954949+0000", 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022234770000000002 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.955036+0000", 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7688999999999993e-05 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.129 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:66 3.as0 3:5ad2eba8:::smithi11731613-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.946822+0000", 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.139744271, 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011586430999999999, 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 66 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.130 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946822+0000", 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946822+0000", 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.131 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946821+0000", 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946824+0000", 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0060000000000001e-06 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946832+0000", 2024-09-16T06:48:23.132 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3429999999999998e-06 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.946839+0000", 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3780000000000003e-06 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949720+0000", 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028810300000000001 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949819+0000", 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8673999999999997e-05 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.955533+0000", 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005714495 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.955650+0000", 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011726300000000001 2024-09-16T06:48:23.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.958408+0000", 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027579309999999999 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:74 3.as0 3:5d679340:::smithi11731613-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.124289944, 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026692272999999999, 2024-09-16T06:48:23.135 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 74 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962275+0000", 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.137 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962280+0000", 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1300000000000003e-06 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962286+0000", 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4529999999999999e-06 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962292+0000", 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5849999999999999e-06 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964248+0000", 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001956667 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964344+0000", 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6174000000000004e-05 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964765+0000", 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042082000000000003 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965862+0000", 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0010968289999999999 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988875+0000", 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023013051 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988969+0000", 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3350000000000006e-05 2024-09-16T06:48:23.140 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:75 3.as0 3:5d679340:::smithi11731613-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.124261632, 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033077179999999998, 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.141 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 75 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.142 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962304+0000", 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962308+0000", 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0600000000000001e-06 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962314+0000", 2024-09-16T06:48:23.143 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.088e-06 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962319+0000", 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1120000000000004e-06 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964978+0000", 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026582540000000001 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.144 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965065+0000", 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7108999999999996e-05 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989270+0000", 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024205503 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989359+0000", 2024-09-16T06:48:23.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9204999999999997e-05 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995382+0000", 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060223200000000003 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=11, last_complete=35'6, committed=1, applied=1))", 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.12052597399999999, 2024-09-16T06:48:23.146 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022946498999999999, 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966039+0000", 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966044+0000", 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6959999999999996e-06 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988553+0000", 2024-09-16T06:48:23.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022508866999999998 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988563+0000", 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0373e-05 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988804+0000", 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000240289 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988987+0000", 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018304199999999999 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:89 3.as0 3:57ffea71:::smithi11731613-30:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.982929+0000", 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.103637281, 2024-09-16T06:48:23.150 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0079098120000000004, 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 89 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982929+0000", 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.151 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982929+0000", 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982928+0000", 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982933+0000", 2024-09-16T06:48:23.152 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3780000000000003e-06 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982947+0000", 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4154999999999999e-05 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982953+0000", 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2179999999999996e-06 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.153 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985021+0000", 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020675979999999999 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985129+0000", 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000108831 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985618+0000", 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000488149 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988519+0000", 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0029019150000000001 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990758+0000", 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022381380000000002 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990839+0000", 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1151999999999998e-05 2024-09-16T06:48:23.155 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:90 3.as0 3:57ffea71:::smithi11731613-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.982958+0000", 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.10360810099999999, 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012877738999999999, 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.156 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 90 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982958+0000", 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982958+0000", 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982958+0000", 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982960+0000", 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5639999999999999e-06 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982967+0000", 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3679999999999998e-06 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.982973+0000", 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2199999999999997e-06 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985971+0000", 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0029986320000000002 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.986204+0000", 2024-09-16T06:48:23.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023217499999999999 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.991178+0000", 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0049739179999999999 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.991258+0000", 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0146000000000001e-05 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995836+0000", 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0045781559999999999 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:95 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.161 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.094377056000000001, 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042547695000000003, 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 95 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992188+0000", 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992194+0000", 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8760000000000002e-06 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992204+0000", 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.011e-05 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992212+0000", 2024-09-16T06:48:23.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1890000000000001e-06 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993832+0000", 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001620565 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993886+0000", 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4021999999999999e-05 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994477+0000", 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059046499999999998 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995906+0000", 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014291919999999999 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034552+0000", 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038646106999999999 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034737+0000", 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018489900000000001 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.167 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:96 3.as0 3:5bc02ad9:::smithi11731613-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.094348073000000005, 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.058410493000000001, 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 96 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992221+0000", 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2899999999999998e-06 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992229+0000", 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3080000000000008e-06 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992236+0000", 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.528e-06 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994995+0000", 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027588700000000001 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995064+0000", 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9758000000000005e-05 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035940+0000", 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040876066000000003 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036021+0000", 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0243999999999999e-05 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050629+0000", 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014608009999999999 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.172 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=18, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.091266324999999995, 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040520444000000003, 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995299+0000", 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995304+0000", 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5680000000000001e-06 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034371+0000", 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039067118999999997 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034380+0000", 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7949999999999996e-06 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034448+0000", 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8244000000000006e-05 2024-09-16T06:48:23.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035820+0000", 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001372306 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpReadReply(3.as0 35/32 ECSubReadReply(tid=20, attrs_read=0))", 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.036901+0000", 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.049665099999999997, 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013737132000000001, 2024-09-16T06:48:23.176 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036901+0000", 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036901+0000", 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036900+0000", 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036927+0000", 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7025999999999998e-05 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050099+0000", 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013171519 2024-09-16T06:48:23.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050108+0000", 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0489999999999993e-06 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050453+0000", 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00034508199999999997 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050638+0000", 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018526999999999999 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpRead(3.ds1 35/32 ECSubRead(tid=11, to_read={3:bbfb3f8a:::smithi11731613-42:head=0,4096,0}, subchunks={3:bbfb3f8a:::smithi11731613-42:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.050064+0000", 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.036502026999999999, 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014089427, 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050064+0000", 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050064+0000", 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050064+0000", 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967296 2024-09-16T06:48:23.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050072+0000", 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9200000000000004e-06 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063831+0000", 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013759349000000001 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063839+0000", 2024-09-16T06:48:23.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0090000000000007e-06 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063899+0000", 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9453000000000002e-05 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064154+0000", 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025493199999999998 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:142 3.as0 3:50fa89c4:::smithi11731613-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.060679+0000", 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.025887349, 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0080520230000000002, 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 142 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060679+0000", 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060679+0000", 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060676+0000", 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:23.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060932+0000", 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025555199999999998 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060955+0000", 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3425e-05 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060968+0000", 2024-09-16T06:48:23.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2863e-05 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.061027+0000", 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9196e-05 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.061191+0000", 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016333 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.062026+0000", 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000835894 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063553+0000", 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015261949999999999 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068663+0000", 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051107189999999997 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068731+0000", 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7554999999999999e-05 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:23.191 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:48:23.191 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:23.191+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:23.292 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:23.287+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:23.358 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:23.359 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:23.359 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:23.359 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:23.372 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:48:23.393 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:23.396+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:23.401 INFO:tasks.rados.rados.0.smithi117.stdout:45: finishing write tid 3 to smithi11731613-45 2024-09-16T06:48:23.401 INFO:tasks.rados.rados.0.smithi117.stdout:45: oid 45 updating version 0 to 10 2024-09-16T06:48:23.401 INFO:tasks.rados.rados.0.smithi117.stdout:45: oid 45 version 10 is already newer than 9 2024-09-16T06:48:23.401 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 45 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-16T06:48:23.401 INFO:tasks.rados.rados.0.smithi117.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-16T06:48:23.446 INFO:tasks.rados.rados.0.smithi117.stdout:48: finishing write tid 3 to smithi11731613-48 2024-09-16T06:48:23.449 INFO:tasks.rados.rados.0.smithi117.stdout:48: finishing write tid 4 to smithi11731613-48 2024-09-16T06:48:23.450 INFO:tasks.rados.rados.0.smithi117.stdout:48: oid 48 updating version 0 to 11 2024-09-16T06:48:23.450 INFO:tasks.rados.rados.0.smithi117.stdout:48: oid 48 updating version 11 to 12 2024-09-16T06:48:23.450 INFO:tasks.rados.rados.0.smithi117.stdout:48: oid 48 updating version 12 to 13 2024-09-16T06:48:23.450 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 48 v 13 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-16T06:48:23.450 INFO:tasks.rados.rados.0.smithi117.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-16T06:48:23.493 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:23.493+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:23.498 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:23.498 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:23.498 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:23.498 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:23.498 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:33: done (8 left) 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:43: done (7 left) 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:45: done (6 left) 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:48: done (5 left) 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:49: done (4 left) 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:53: done (3 left) 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 47 current snap is 0 2024-09-16T06:48:23.499 INFO:tasks.rados.rados.0.smithi117.stdout:54: seq_num 52 ranges {786432=507904,1294336=376832} 2024-09-16T06:48:23.506 INFO:tasks.rados.rados.0.smithi117.stdout:54: writing smithi11731613-47 from 786432 to 1294336 tid 1 2024-09-16T06:48:23.509 INFO:tasks.rados.rados.0.smithi117.stdout:54: writing smithi11731613-47 from 1294336 to 1671168 tid 2 2024-09-16T06:48:23.509 INFO:tasks.rados.rados.0.smithi117.stdout:55: copy_from oid 46 from oid 2 current snap is 0 2024-09-16T06:48:23.509 INFO:tasks.rados.rados.0.smithi117.stdout:56: snap_create 2024-09-16T06:48:23.514 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:48:23.516 INFO:tasks.rados.rados.0.smithi117.stdout:54: finishing write tid 1 to smithi11731613-47 2024-09-16T06:48:23.517 INFO:tasks.rados.rados.0.smithi117.stdout:50: finishing write tid 3 to smithi11731613-50 2024-09-16T06:48:23.518 INFO:tasks.rados.rados.0.smithi117.stdout:54: finishing write tid 2 to smithi11731613-47 2024-09-16T06:48:23.518 INFO:tasks.rados.rados.0.smithi117.stdout:54: finishing write tid 3 to smithi11731613-47 2024-09-16T06:48:23.521 INFO:tasks.rados.rados.0.smithi117.stdout:50: finishing write tid 4 to smithi11731613-50 2024-09-16T06:48:23.522 INFO:tasks.rados.rados.0.smithi117.stdout:50: oid 50 updating version 0 to 9 2024-09-16T06:48:23.522 INFO:tasks.rados.rados.0.smithi117.stdout:50: oid 50 version 9 is already newer than 8 2024-09-16T06:48:23.522 INFO:tasks.rados.rados.0.smithi117.stdout:50: oid 50 updating version 9 to 10 2024-09-16T06:48:23.522 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 50 v 10 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T06:48:23.522 INFO:tasks.rados.rados.0.smithi117.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-16T06:48:23.522 INFO:tasks.rados.rados.0.smithi117.stdout:55: finishing copy_from racing read to smithi11731613-46 2024-09-16T06:48:23.523 INFO:tasks.rados.rados.0.smithi117.stdout:54: finishing write tid 4 to smithi11731613-47 2024-09-16T06:48:23.523 INFO:tasks.rados.rados.0.smithi117.stdout:54: oid 47 updating version 0 to 9 2024-09-16T06:48:23.523 INFO:tasks.rados.rados.0.smithi117.stdout:54: oid 47 version 9 is already newer than 8 2024-09-16T06:48:23.523 INFO:tasks.rados.rados.0.smithi117.stdout:54: oid 47 version 9 is already newer than 7 2024-09-16T06:48:23.523 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 47 v 9 (ObjNum 52 snap 0 seq_num 52) dirty exists 2024-09-16T06:48:23.523 INFO:tasks.rados.rados.0.smithi117.stdout:54: left oid 47 (ObjNum 52 snap 0 seq_num 52) 2024-09-16T06:48:23.594 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:23.592+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:23.612 INFO:tasks.rados.rados.0.smithi117.stdout:18: finishing write tid 2 to smithi11731613-18 2024-09-16T06:48:23.639 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:23.639 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:1 3.ds0 3:b273fc7d:::smithi11731613-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.74544438700000004, 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011215246, 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.640 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 1 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.641 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892973+0000", 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893050+0000", 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6996999999999995e-05 2024-09-16T06:48:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893075+0000", 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5406999999999999e-05 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893092+0000", 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6541999999999998e-05 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893183+0000", 2024-09-16T06:48:23.643 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1404999999999996e-05 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893334+0000", 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015069399999999999 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894090+0000", 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00075619499999999998 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.644 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901851+0000", 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0077611449999999997 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012789380000000001 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904191+0000", 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001060636 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.645 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.74531042199999997, 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.646 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.647 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:48:23.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:48:23.649 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.650 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.745226626, 2024-09-16T06:48:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.652 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:48:23.653 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.654 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.655 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.656 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 35/32 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.897843+0000", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.74057676699999997, 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00388602, 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897843+0000", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897843+0000", 2024-09-16T06:48:23.657 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897841+0000", 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897855+0000", 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3592e-05 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897951+0000", 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6062000000000003e-05 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897972+0000", 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0718000000000001e-05 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901681+0000", 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037088490000000002 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901729+0000", 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8739000000000002e-05 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 34/32 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.898567+0000", 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.73985323800000002, 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0032875030000000002, 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.660 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898567+0000", 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898567+0000", 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898566+0000", 2024-09-16T06:48:23.661 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898578+0000", 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2622e-05 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898761+0000", 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018321099999999999 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898773+0000", 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1314e-05 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901812+0000", 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030394979999999999 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901854+0000", 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2153000000000001e-05 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.663 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 34/32 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.900387+0000", 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.73803314900000005, 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0038489940000000001, 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900387+0000", 2024-09-16T06:48:23.664 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900387+0000", 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900387+0000", 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900389+0000", 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3989999999999998e-06 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900523+0000", 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013390599999999999 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900527+0000", 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4059999999999998e-06 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.666 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902894+0000", 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023661659999999998 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904236+0000", 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001342389 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 35/32 ECSubWrite(tid=1, reqid=client.4511.0:23, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:23.667 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.908330+0000", 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.73008991700000003, 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0043693589999999997, 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908330+0000", 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908330+0000", 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908329+0000", 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908359+0000", 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9918000000000001e-05 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908409+0000", 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0216000000000002e-05 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908421+0000", 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1959e-05 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908512+0000", 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0752000000000001e-05 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.670 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.908534+0000", 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2208000000000002e-05 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.912670+0000", 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0041353359999999999 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.912700+0000", 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9980000000000001e-05 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.671 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=2, reqid=client.4511.0:28, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.909098+0000", 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.72932208899999995, 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060853890000000001, 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909098+0000", 2024-09-16T06:48:23.672 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909098+0000", 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909097+0000", 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909168+0000", 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1031000000000005e-05 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909265+0000", 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7632000000000003e-05 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909289+0000", 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3417999999999998e-05 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909357+0000", 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8022999999999996e-05 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909365+0000", 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6540000000000005e-06 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915163+0000", 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0057975659999999997 2024-09-16T06:48:23.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915183+0000", 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0615000000000002e-05 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=3, reqid=client.4511.0:29, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.911582+0000", 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.72683779999999998, 2024-09-16T06:48:23.676 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036202690000000002, 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911582+0000", 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911582+0000", 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.677 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911581+0000", 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911601+0000", 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9480999999999999e-05 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911643+0000", 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1909999999999997e-05 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.678 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911651+0000", 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1300000000000001e-06 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911749+0000", 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8225000000000002e-05 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911773+0000", 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3587999999999998e-05 2024-09-16T06:48:23.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915192+0000", 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003418992 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915203+0000", 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1082e-05 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.680 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:49 3.ds0 3:b2c197ab:::smithi11731613-17:head [append 0~278528,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924020+0000", 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.71440009699999996, 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0063827110000000001, 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 49 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.681 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924020+0000", 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924020+0000", 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924018+0000", 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.682 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924468+0000", 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000450617 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924536+0000", 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7886999999999996e-05 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924557+0000", 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1310000000000001e-05 2024-09-16T06:48:23.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924633+0000", 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5603000000000001e-05 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924778+0000", 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000145461 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.925544+0000", 2024-09-16T06:48:23.684 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00076506700000000003 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927141+0000", 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015972740000000001 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930348+0000", 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0032070139999999998 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930403+0000", 2024-09-16T06:48:23.685 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4920999999999998e-05 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.71382138699999997, 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.686 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:48:23.688 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.689 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.690 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.691 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.71377161899999997, 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.692 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.693 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:48:23.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:48:23.695 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.696 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=7, reqid=client.4511.0:65, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.949399+0000", 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.68902146500000006, 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0032919540000000001, 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949399+0000", 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949399+0000", 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949398+0000", 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949418+0000", 2024-09-16T06:48:23.698 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0656e-05 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949462+0000", 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3875000000000003e-05 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949471+0000", 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3350000000000007e-06 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.699 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949540+0000", 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9053000000000004e-05 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949553+0000", 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3661e-05 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952665+0000", 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00311138 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952691+0000", 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5856000000000001e-05 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=12, reqid=client.4511.0:88, at_version=35'7, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.984691+0000", 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.65372874400000003, 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003662556, 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.701 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984691+0000", 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984691+0000", 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.702 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984690+0000", 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984774+0000", 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.4214000000000004e-05 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984833+0000", 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8431999999999999e-05 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.703 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984846+0000", 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3393e-05 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984900+0000", 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3804999999999997e-05 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984909+0000", 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6150000000000003e-06 2024-09-16T06:48:23.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988327+0000", 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034176409999999999 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988354+0000", 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6991999999999998e-05 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.705 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.6s5 35/32 ECSubWrite(tid=15, reqid=client.4511.0:103, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.007122+0000", 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.63129862299999995, 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036232259999999998, 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007122+0000", 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.706 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007122+0000", 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007120+0000", 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007510+0000", 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039044900000000002 2024-09-16T06:48:23.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007556+0000", 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5454000000000002e-05 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007566+0000", 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0239e-05 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.708 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007647+0000", 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0795000000000001e-05 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.007657+0000", 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0538e-05 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.010719+0000", 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003061154 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.709 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.010745+0000", 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6197e-05 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.60603395400000004, 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.710 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:23.711 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.712 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.713 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:48:23.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.715 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.60547849499999995, 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.716 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:48:23.717 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.718 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.719 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:48:23.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.60542216400000004, 2024-09-16T06:48:23.721 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.722 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:48:23.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:48:23.724 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=22, reqid=client.4511.0:143, at_version=35'12, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.064672+0000", 2024-09-16T06:48:23.726 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.57374810700000001, 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036319730000000001, 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064672+0000", 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064672+0000", 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064671+0000", 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064747+0000", 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5876000000000003e-05 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064791+0000", 2024-09-16T06:48:23.728 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4322e-05 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064801+0000", 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0223e-05 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064858+0000", 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6654999999999999e-05 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.729 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064867+0000", 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.704e-06 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068272+0000", 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034056469999999999 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068304+0000", 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1678999999999999e-05 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.730 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 35/32 ECSubWrite(tid=23, reqid=client.4511.0:144, at_version=35'13, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.065682+0000", 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.57273790499999999, 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052142550000000001, 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065682+0000", 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065682+0000", 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065681+0000", 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065701+0000", 2024-09-16T06:48:23.732 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9198000000000002e-05 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065765+0000", 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4132000000000002e-05 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065773+0000", 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.4980000000000006e-06 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065848+0000", 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4813000000000004e-05 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065859+0000", 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1292e-05 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070859+0000", 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0049992329999999996 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070896+0000", 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.7902999999999999e-05 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.735 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:23.737 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:48:23.737 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:23.736+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:23.837 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:23.836+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:23.855 INFO:tasks.rados.rados.0.smithi117.stdout:52: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-16T06:48:23.895 INFO:tasks.rados.rados.0.smithi117.stdout:50: done (5 left) 2024-09-16T06:48:23.895 INFO:tasks.rados.rados.0.smithi117.stdout:52: done (4 left) 2024-09-16T06:48:23.896 INFO:tasks.rados.rados.0.smithi117.stdout:54: done (3 left) 2024-09-16T06:48:23.896 INFO:tasks.rados.rados.0.smithi117.stdout:56: done (2 left) 2024-09-16T06:48:23.896 INFO:tasks.rados.rados.0.smithi117.stdout:57: snap_create 2024-09-16T06:48:23.896 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:48:23.899 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:23.899 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:23.900 INFO:tasks.thrashosds.thrasher:Removing osd 13, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-16T06:48:23.900 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 13 2024-09-16T06:48:23.908 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:23.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:163 3.cs0 3:31c9f1c0:::smithi11731613-46:head [copy-from ver 2 in=54b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.39863774699999999, 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39866272600000002, 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 163 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:23.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:23.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512085+0000", 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8870000000000002e-06 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512091+0000", 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3539999999999996e-06 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:23.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512096+0000", 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8029999999999996e-06 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513808+0000", 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001712199 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513857+0000", 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8649000000000001e-05 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:23.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.524232+0000", 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010375422 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.527189+0000", 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002956939 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 1 2024-09-16T06:48:23.913 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:23.922 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:48:23.938 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:23.936+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:24.039 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:24.040+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:24.043 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:24.043 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:24.043 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:24.043 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:24.043 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:24.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:48:24.140 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:24.140+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:13 3.fs0 3:fe6e9dcb:::smithi11731613-5:head [append 0~81920,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895200+0000", 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2807930249999999, 2024-09-16T06:48:24.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024214652, 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 13 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895200+0000", 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895200+0000", 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895197+0000", 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895323+0000", 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012593700000000001 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895344+0000", 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0755999999999999e-05 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895360+0000", 2024-09-16T06:48:24.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6347000000000001e-05 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896017+0000", 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00065678899999999996 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896210+0000", 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00019296999999999999 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897228+0000", 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001018363 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911201+0000", 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013972581 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919346+0000", 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0081450059999999998 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919414+0000", 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8468999999999998e-05 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:14 3.fs0 3:fe6e9dcb:::smithi11731613-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895390+0000", 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.280602375, 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027519056, 2024-09-16T06:48:24.184 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.185 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.185 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.185 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.185 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.185 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 14 2024-09-16T06:48:24.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.188 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895390+0000", 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895390+0000", 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895389+0000", 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895396+0000", 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6279999999999999e-06 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895408+0000", 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1610000000000001e-05 2024-09-16T06:48:24.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895416+0000", 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5790000000000004e-06 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897891+0000", 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024744490000000001 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898011+0000", 2024-09-16T06:48:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012032900000000001 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898652+0000", 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00064155199999999999 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911204+0000", 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012552001 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.922586+0000", 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011381673 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.922909+0000", 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032314300000000002 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.193 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:15 3.fs0 3:fe6e9dcb:::smithi11731613-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895424+0000", 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.280568441, 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033755443000000003, 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 15 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.194 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895424+0000", 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895424+0000", 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895423+0000", 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895428+0000", 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1500000000000001e-06 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895438+0000", 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0497e-05 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895445+0000", 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2660000000000004e-06 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898920+0000", 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034744609999999999 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.899038+0000", 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000118103 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.923251+0000", 2024-09-16T06:48:24.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024213373999999999 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.923421+0000", 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016966799999999999 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.929180+0000", 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0057586740000000001 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:19 3.fs0 3:fa1c58b4:::smithi11731613-7:head [append 0~114688,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895453+0000", 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2805398450000001, 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031813735000000003, 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 19 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.199 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895453+0000", 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895453+0000", 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895452+0000", 2024-09-16T06:48:24.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895633+0000", 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000180905 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895647+0000", 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3987e-05 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895657+0000", 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0040999999999999e-05 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.899080+0000", 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034227849999999998 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.899249+0000", 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016941699999999999 2024-09-16T06:48:24.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900154+0000", 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090474399999999997 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911262+0000", 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011108386 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926372+0000", 2024-09-16T06:48:24.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015110006 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927266+0000", 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00089414599999999996 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:20 3.fs0 3:fa1c58b4:::smithi11731613-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895667+0000", 2024-09-16T06:48:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2803257889999999, 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034325209000000002, 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 20 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895667+0000", 2024-09-16T06:48:24.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895667+0000", 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895666+0000", 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895671+0000", 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9010000000000002e-06 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895681+0000", 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9760000000000007e-06 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895693+0000", 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2534000000000001e-05 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909215+0000", 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013521348000000001 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909779+0000", 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00056460899999999999 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910929+0000", 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011494649999999999 2024-09-16T06:48:24.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913071+0000", 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021418029999999998 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.929931+0000", 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016859986 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.929992+0000", 2024-09-16T06:48:24.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.1470999999999995e-05 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:21 3.fs0 3:fa1c58b4:::smithi11731613-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2802918999999999, 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.036267527000000001, 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.210 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 21 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895700+0000", 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895704+0000", 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.048e-06 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895714+0000", 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8509999999999994e-06 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895721+0000", 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2339999999999997e-06 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913213+0000", 2024-09-16T06:48:24.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017491642000000002 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913291+0000", 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8053999999999995e-05 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930092+0000", 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016801100999999999 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930264+0000", 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017232 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.931968+0000", 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017039620000000001 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:25 3.fs0 3:f3961c35:::smithi11731613-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:24.215 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.280264091, 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.036948371000000001, 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 25 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:24.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895853+0000", 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000125451 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895874+0000", 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0346000000000001e-05 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895883+0000", 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0999999999999993e-06 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913349+0000", 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017466612999999999 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913577+0000", 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022744 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914548+0000", 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00097095700000000005 2024-09-16T06:48:24.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917700+0000", 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031521740000000002 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.932545+0000", 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014845191000000001 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.932677+0000", 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013169899999999999 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:26 3.fs0 3:f3961c35:::smithi11731613-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2801007049999999, 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.037776441000000001, 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.221 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 26 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:48:24.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895891+0000", 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895896+0000", 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9599999999999999e-06 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895906+0000", 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9969999999999993e-06 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895913+0000", 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.08e-06 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915339+0000", 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.019426282 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915437+0000", 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7822e-05 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916103+0000", 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00066603099999999998 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917703+0000", 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015995969999999999 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.933600+0000", 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015897047000000001 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.933668+0000", 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8465000000000003e-05 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.226 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:27 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2800718170000001, 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039993954999999998, 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 27 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895920+0000", 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895924+0000", 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.506e-06 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895933+0000", 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.876e-06 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895941+0000", 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8229999999999995e-06 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916469+0000", 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.020528105000000001 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916546+0000", 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6815000000000002e-05 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.934316+0000", 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017769672 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.934388+0000", 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1753000000000002e-05 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.935915+0000", 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015271079999999999 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:63 3.fs0 3:fbae4168:::smithi11731613-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.941801+0000", 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2341919109999999, 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010096476, 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 63 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.941801+0000", 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.941801+0000", 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.941800+0000", 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.941822+0000", 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2435000000000001e-05 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.941837+0000", 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5173e-05 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.941849+0000", 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1499e-05 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.944338+0000", 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024891290000000001 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.944424+0000", 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6373999999999997e-05 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950371+0000", 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0059465610000000004 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950448+0000", 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7002999999999994e-05 2024-09-16T06:48:24.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.951897+0000", 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014493869999999999 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:123 3.fs0 3:f943c919:::smithi11731613-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.027793+0000", 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.148200007, 2024-09-16T06:48:24.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0096512309999999997, 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 123 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.027793+0000", 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.027793+0000", 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.027792+0000", 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.027795+0000", 2024-09-16T06:48:24.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0860000000000002e-06 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.027803+0000", 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9990000000000008e-06 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.027808+0000", 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6210000000000003e-06 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.030378+0000", 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0025708660000000002 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.030495+0000", 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011699799999999999 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035572+0000", 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050764989999999999 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035651+0000", 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.928e-05 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.037444+0000", 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017925370000000001 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.242 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:132 3.cs0 3:3ac5e123:::smithi11731613-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.042642+0000", 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.1333505960000001, 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0096718470000000008, 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 132 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.042642+0000", 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.042642+0000", 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.042642+0000", 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.042645+0000", 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3510000000000002e-06 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.042652+0000", 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7360000000000004e-06 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.042671+0000", 2024-09-16T06:48:24.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9754000000000001e-05 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045051+0000", 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023796759999999998 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045140+0000", 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8764999999999995e-05 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050091+0000", 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00495123 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050167+0000", 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5766999999999995e-05 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052314+0000", 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021470479999999999 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.247 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:138 3.cs0 3:31c9f1c0:::smithi11731613-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.052927+0000", 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.1230652059999999, 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011788320999999999, 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.248 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 138 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052927+0000", 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052927+0000", 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052927+0000", 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052929+0000", 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9300000000000002e-06 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052934+0000", 2024-09-16T06:48:24.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5779999999999998e-06 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.052941+0000", 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2530000000000001e-06 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.054932+0000", 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019909760000000002 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.055006+0000", 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4647000000000005e-05 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.061610+0000", 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066034229999999998 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.061696+0000", 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6468e-05 2024-09-16T06:48:24.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064716+0000", 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003019463 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:140 3.cs0 3:3e2de01d:::smithi11731613-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.057973+0000", 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.1180197350000001, 2024-09-16T06:48:24.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010709316999999999, 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 140 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.057973+0000", 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.057973+0000", 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.057972+0000", 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.057977+0000", 2024-09-16T06:48:24.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9429999999999999e-06 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.057986+0000", 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3850000000000008e-06 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.057994+0000", 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1879999999999996e-06 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.059168+0000", 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011743210000000001 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.059278+0000", 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109685 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.059783+0000", 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00050511299999999996 2024-09-16T06:48:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060193+0000", 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00041051999999999999 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068600+0000", 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084062849999999995 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068682+0000", 2024-09-16T06:48:24.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2637000000000002e-05 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:141 3.cs0 3:3e2de01d:::smithi11731613-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.058000+0000", 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.1179921740000001, 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013535953, 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 141 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.058000+0000", 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.058000+0000", 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.058000+0000", 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.058003+0000", 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2310000000000001e-06 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.058011+0000", 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7559999999999999e-06 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.058016+0000", 2024-09-16T06:48:24.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2460000000000003e-06 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.059998+0000", 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019820549999999999 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.060081+0000", 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3200000000000003e-05 2024-09-16T06:48:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.069071+0000", 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0089898140000000005 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.069155+0000", 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3657000000000002e-05 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.071536+0000", 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023815910000000002 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 35/32 ECSubWrite(tid=35, reqid=client.4511.0:152, at_version=35'10, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.098152856, 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.44133388099999998, 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.265 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077839+0000", 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077850+0000", 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0505e-05 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515878+0000", 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.43802848300000002 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515883+0000", 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1780000000000002e-06 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515923+0000", 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9866000000000001e-05 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515930+0000", 2024-09-16T06:48:24.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.224e-06 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519096+0000", 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031664449999999999 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519174+0000", 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7590999999999996e-05 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.269 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:159 3.cs0 3:3e2de01d:::smithi11731613-47:head [append 0~507904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.509200+0000", 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.66679254099999996, 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0097287980000000003, 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 159 2024-09-16T06:48:24.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509200+0000", 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509200+0000", 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.271 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509198+0000", 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509459+0000", 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026146200000000001 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509486+0000", 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6567000000000002e-05 2024-09-16T06:48:24.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509498+0000", 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1973e-05 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509560+0000", 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2137000000000005e-05 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.509674+0000", 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000114046 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.510567+0000", 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00089325099999999996 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.511680+0000", 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001113263 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.518879+0000", 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0071983780000000001 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.518929+0000", 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0087999999999999e-05 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.275 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:162 3.cs0 3:3e2de01d:::smithi11731613-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512050+0000", 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.66394266999999996, 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013796572999999999, 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 162 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.276 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512050+0000", 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512050+0000", 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512049+0000", 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512052+0000", 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6680000000000001e-06 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512059+0000", 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6669999999999999e-06 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512064+0000", 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0000000000000004e-06 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513569+0000", 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015054070000000001 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513756+0000", 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018694800000000001 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.521378+0000", 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0076216979999999997 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.521461+0000", 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3467999999999994e-05 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.525846+0000", 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0043851940000000002 2024-09-16T06:48:24.280 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:164 3.cs0 3:31c9f1c0:::smithi11731613-46:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512101+0000", 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.66389178100000001, 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013135302999999999, 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.281 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 164 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512101+0000", 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512101+0000", 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512101+0000", 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512102+0000", 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8330000000000001e-06 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512108+0000", 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0739999999999998e-06 2024-09-16T06:48:24.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512115+0000", 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0859999999999996e-06 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.514036+0000", 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001920513 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:24.284 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.514089+0000", 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2808e-05 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.525089+0000", 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010999833000000001 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.525158+0000", 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9369999999999995e-05 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.525236+0000", 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8084999999999995e-05 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.64730915300000003, 2024-09-16T06:48:24.286 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.394952786, 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528687+0000", 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0810000000000004e-06 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923559+0000", 2024-09-16T06:48:24.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39487182199999998 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923572+0000", 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3406e-05 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923624+0000", 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1989e-05 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923636+0000", 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1990999999999999e-05 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.290 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:24.291 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-16T06:48:24.208+0000 7fce2dab6640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-16T06:48:24.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:48:24.293 INFO:tasks.rados.rados.0.smithi117.stdout:18: finishing write tid 3 to smithi11731613-18 2024-09-16T06:48:24.293 INFO:tasks.rados.rados.0.smithi117.stdout:18: oid 18 updating version 0 to 1 2024-09-16T06:48:24.293 INFO:tasks.rados.rados.0.smithi117.stdout:18: oid 18 updating version 1 to 2 2024-09-16T06:48:24.293 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 18 v 2 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-16T06:48:24.293 INFO:tasks.rados.rados.0.smithi117.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-16T06:48:24.293 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:24.289+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:24.317 INFO:tasks.rados.rados.0.smithi117.stdout:55: finishing copy_from to smithi11731613-46 2024-09-16T06:48:24.317 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 46 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:24.393 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:24.392+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:24.436 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:24.445 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:24.445 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:24.445 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:24.445 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:24.459 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:48:24.493 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:24.492+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:24.583 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:24.583 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:24.583 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:24.584 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:24.584 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:24.594 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:24.592+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:24.596 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:24.596 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:24.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:48:24.620 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:24.620 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:24.645 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:24.646 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:24.673 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:24.673 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:24.695 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:24.696+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:24.697 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:24.697 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 35/32 ECSubWrite(tid=1, reqid=client.4511.0:4, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.896676+0000", 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.8276283639999999, 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0033975860000000002, 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.724 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896676+0000", 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896676+0000", 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.725 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896675+0000", 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896720+0000", 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4749000000000001e-05 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896814+0000", 2024-09-16T06:48:24.726 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4293e-05 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896829+0000", 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5214999999999999e-05 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896905+0000", 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5617000000000005e-05 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.896926+0000", 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1226999999999999e-05 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900004+0000", 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030771819999999999 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900073+0000", 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9657000000000001e-05 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.728 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.897946+0000", 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.82635828, 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0069738930000000001, 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.729 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897946+0000", 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897946+0000", 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897945+0000", 2024-09-16T06:48:24.730 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897973+0000", 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8302000000000001e-05 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898056+0000", 2024-09-16T06:48:24.731 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2748000000000001e-05 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898068+0000", 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2767e-05 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898146+0000", 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7732999999999996e-05 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.732 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898159+0000", 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3233e-05 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904857+0000", 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066979099999999996 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904920+0000", 2024-09-16T06:48:24.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2213999999999998e-05 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=2, reqid=client.4511.0:28, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.909500+0000", 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.8148039279999999, 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031118399999999998, 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909500+0000", 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909500+0000", 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909500+0000", 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967296 2024-09-16T06:48:24.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909566+0000", 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6118999999999995e-05 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909631+0000", 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4703999999999996e-05 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909642+0000", 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1426e-05 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909702+0000", 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9953e-05 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909710+0000", 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9200000000000004e-06 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.737 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.912585+0000", 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028751129999999999 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.912612+0000", 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6815999999999999e-05 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.738 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 35/32 ECSubWrite(tid=1, reqid=client.4511.0:23, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.910679+0000", 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.8136246979999999, 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035740250000000002, 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910679+0000", 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.739 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910679+0000", 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910678+0000", 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910698+0000", 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9627999999999999e-05 2024-09-16T06:48:24.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910738+0000", 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0266999999999999e-05 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910747+0000", 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.4149999999999999e-06 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910819+0000", 2024-09-16T06:48:24.741 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2383000000000001e-05 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910832+0000", 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2819e-05 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914214+0000", 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003381659 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914253+0000", 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9725000000000003e-05 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=3, reqid=client.4511.0:29, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.911595+0000", 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.812708454, 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024523539999999999, 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.743 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911595+0000", 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911595+0000", 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911595+0000", 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911615+0000", 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9939999999999999e-05 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911671+0000", 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5936000000000001e-05 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911680+0000", 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.5740000000000002e-06 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911726+0000", 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5612999999999998e-05 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911735+0000", 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9330000000000007e-06 2024-09-16T06:48:24.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914013+0000", 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022785869999999999 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914048+0000", 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4604000000000003e-05 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 35/32 ECSubWrite(tid=2, reqid=client.4511.0:31, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.913663+0000", 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.810641197, 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028998679999999999, 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913663+0000", 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913663+0000", 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913662+0000", 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.749 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913758+0000", 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6088999999999994e-05 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913799+0000", 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0856000000000003e-05 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913807+0000", 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.4630000000000001e-06 2024-09-16T06:48:24.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913854+0000", 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6677e-05 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913864+0000", 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9869999999999994e-06 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916521+0000", 2024-09-16T06:48:24.751 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026572200000000001 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916563+0000", 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1303000000000001e-05 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 35/32 ECSubWrite(tid=0, reqid=client.4511.0:46, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.918940+0000", 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.805363915, 2024-09-16T06:48:24.752 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028981020000000001, 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918940+0000", 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918940+0000", 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918939+0000", 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919272+0000", 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000333188 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919367+0000", 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4815000000000006e-05 2024-09-16T06:48:24.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919386+0000", 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9244000000000001e-05 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919452+0000", 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5488999999999996e-05 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919461+0000", 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.986e-06 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921802+0000", 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023410250000000001 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921838+0000", 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6474999999999999e-05 2024-09-16T06:48:24.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 35/32 ECSubWrite(tid=1, reqid=client.4511.0:47, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.919279+0000", 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.805024475, 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0025952670000000001, 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919279+0000", 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919279+0000", 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919279+0000", 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.758 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919301+0000", 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2806000000000002e-05 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919428+0000", 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012629899999999999 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919441+0000", 2024-09-16T06:48:24.759 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3078000000000001e-05 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919820+0000", 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037909599999999999 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.919833+0000", 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3587000000000001e-05 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.760 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921855+0000", 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020219880000000002 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921875+0000", 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9375e-05 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.761 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=6, reqid=client.4511.0:64, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.948060+0000", 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.776243794, 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0040514879999999998, 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948060+0000", 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948060+0000", 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948059+0000", 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948166+0000", 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107094 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.763 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948213+0000", 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6619999999999997e-05 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948222+0000", 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3689999999999996e-06 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948289+0000", 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6501999999999994e-05 2024-09-16T06:48:24.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.948302+0000", 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3392e-05 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952076+0000", 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037738450000000001 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952112+0000", 2024-09-16T06:48:24.765 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5698999999999998e-05 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 35/32 ECSubWrite(tid=9, reqid=client.4511.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.949482+0000", 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7748216810000002, 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028844809999999999, 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.766 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949482+0000", 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949482+0000", 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.767 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949481+0000", 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949497+0000", 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5447e-05 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949542+0000", 2024-09-16T06:48:24.768 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4852999999999999e-05 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949552+0000", 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0808999999999999e-05 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949611+0000", 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8377e-05 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949620+0000", 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9190000000000005e-06 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952332+0000", 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027121049999999998 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952367+0000", 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5012999999999998e-05 2024-09-16T06:48:24.770 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=7, reqid=client.4511.0:65, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.949606+0000", 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7746981239999999, 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028273859999999999, 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.771 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949606+0000", 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949606+0000", 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949605+0000", 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.772 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949624+0000", 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8794e-05 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949668+0000", 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4155e-05 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949676+0000", 2024-09-16T06:48:24.773 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7579999999999992e-06 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949766+0000", 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0283000000000003e-05 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949779+0000", 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2894999999999999e-05 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.774 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952408+0000", 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002628858 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952433+0000", 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5398999999999999e-05 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 35/32 ECSubWrite(tid=8, reqid=client.4511.0:70, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:24.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.960492+0000", 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7638119890000001, 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027909150000000001, 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960492+0000", 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.776 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960492+0000", 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960491+0000", 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960618+0000", 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012685600000000001 2024-09-16T06:48:24.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960662+0000", 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4274000000000003e-05 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960674+0000", 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1924e-05 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.778 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960731+0000", 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7262000000000002e-05 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.960743+0000", 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1231e-05 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.963248+0000", 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0025055339999999998 2024-09-16T06:48:24.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.963283+0000", 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4761000000000001e-05 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:85 3.5s0 3:af79444c:::smithi11731613-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.983710+0000", 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7405942940000001, 2024-09-16T06:48:24.780 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.006079175, 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 85 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.983710+0000", 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.983710+0000", 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.983708+0000", 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984127+0000", 2024-09-16T06:48:24.782 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00041826 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984139+0000", 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2675e-05 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984147+0000", 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0989999999999999e-06 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984216+0000", 2024-09-16T06:48:24.783 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8924000000000006e-05 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984374+0000", 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015792200000000001 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985825+0000", 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001450331 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988717+0000", 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028918870000000001 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989713+0000", 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00099672999999999993 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989789+0000", 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5542000000000003e-05 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.785 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:86 3.5s0 3:af79444c:::smithi11731613-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.984167+0000", 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.740136729, 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0059986040000000003, 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 86 2024-09-16T06:48:24.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984167+0000", 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984167+0000", 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984166+0000", 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984171+0000", 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0669999999999999e-06 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984178+0000", 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3239999999999999e-06 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984182+0000", 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5290000000000002e-06 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.986462+0000", 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022802090000000001 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.986519+0000", 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6132999999999999e-05 2024-09-16T06:48:24.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.987064+0000", 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00054519200000000001 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988721+0000", 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001657438 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990112+0000", 2024-09-16T06:48:24.790 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013912169999999999 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990166+0000", 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3412000000000003e-05 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:87 3.5s0 3:af79444c:::smithi11731613-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:24.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.984219+0000", 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7400849869999999, 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084208679999999998, 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 87 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984219+0000", 2024-09-16T06:48:24.792 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984219+0000", 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984218+0000", 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984223+0000", 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3919999999999996e-06 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984232+0000", 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4870000000000008e-06 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984239+0000", 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7399999999999998e-06 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.794 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.987296+0000", 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030572870000000001 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.987421+0000", 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000124315 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990433+0000", 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030123810000000002 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.795 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990508+0000", 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5105999999999995e-05 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992640+0000", 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002131834 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=12, reqid=client.4511.0:88, at_version=35'7, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:48:24.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.984838+0000", 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7394662190000001, 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0038634009999999998, 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984838+0000", 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984838+0000", 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984837+0000", 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984939+0000", 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000102116 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984984+0000", 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5578000000000003e-05 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984992+0000", 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6750000000000002e-06 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985058+0000", 2024-09-16T06:48:24.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6344e-05 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985070+0000", 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1745e-05 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988675+0000", 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036047190000000002 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988701+0000", 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6344000000000001e-05 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=18, reqid=client.4511.0:95, at_version=35'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.994568+0000", 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.7297356480000001, 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002752235, 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.801 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994568+0000", 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994568+0000", 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994568+0000", 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994584+0000", 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6290000000000002e-05 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994655+0000", 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1557999999999995e-05 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.803 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994665+0000", 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1730000000000001e-06 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994871+0000", 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000206689 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994879+0000", 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0989999999999999e-06 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.997294+0000", 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024144510000000002 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.997321+0000", 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6667999999999999e-05 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.805 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=22, reqid=client.4511.0:143, at_version=35'12, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.064626+0000", 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.65967825, 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003668162, 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064626+0000", 2024-09-16T06:48:24.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064626+0000", 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064625+0000", 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064672+0000", 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6832999999999997e-05 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064721+0000", 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9492e-05 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064734+0000", 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2784e-05 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064784+0000", 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9945999999999999e-05 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064792+0000", 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.388e-06 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068267+0000", 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003475105 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068294+0000", 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6554e-05 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=23, reqid=client.4511.0:144, at_version=35'13, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:48:24.810 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.065744+0000", 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.6585595290000001, 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0025667530000000002, 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065744+0000", 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065744+0000", 2024-09-16T06:48:24.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065744+0000", 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065762+0000", 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8108e-05 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065811+0000", 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9084e-05 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065820+0000", 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.5820000000000008e-06 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065867+0000", 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6394000000000003e-05 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065876+0000", 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0010000000000007e-06 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068301+0000", 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024251199999999998 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.068311+0000", 2024-09-16T06:48:24.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0220999999999999e-05 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 35/32 ECSubWrite(tid=24, reqid=client.4511.0:144, at_version=0'0, trim_to=0'0, roll_forward_to=35'13))", 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.448421+0000", 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2758832330000001, 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034276689999999999, 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448421+0000", 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448421+0000", 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448419+0000", 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448435+0000", 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5152e-05 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448500+0000", 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5841999999999997e-05 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448513+0000", 2024-09-16T06:48:24.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2972e-05 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448655+0000", 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000141948 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448668+0000", 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.274e-05 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:24.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.451820+0000", 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031517020000000001 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.451848+0000", 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8674e-05 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:24.819 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:24.821 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:24.821 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:24.822 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:24.822 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:24.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:48:24.823 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:24.823 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:24.824 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:24.824 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:24.824 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:24.820+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:24.843 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:24.843 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:24.870 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:24.870 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:24.898 INFO:teuthology.orchestra.run.smithi100.stderr:marked out osd.13. 2024-09-16T06:48:24.898 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:24.898 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:18: done (2 left) 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:55: done (1 left) 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:57: done (0 left) 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:58: rollback oid 38 current snap is 2 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 38 to 0 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 48 current snap is 2 2024-09-16T06:48:24.899 INFO:tasks.rados.rados.0.smithi117.stdout:59: seq_num 53 ranges {802816=524288,1327104=376832} 2024-09-16T06:48:24.913 INFO:tasks.rados.rados.0.smithi117.stdout:59: writing smithi11731613-48 from 802816 to 1327104 tid 1 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:59: writing smithi11731613-48 from 1327104 to 1703936 tid 2 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:60: copy_from oid 3 from oid 6 current snap is 2 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:61: rollback oid 5 current snap is 2 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 5 to 0 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:62: read oid 31 snap -1 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:62: expect (ObjNum 30 snap 0 seq_num 30) 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:63: read oid 37 snap 0 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:63: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 10 current snap is 2 2024-09-16T06:48:24.916 INFO:tasks.rados.rados.0.smithi117.stdout:64: seq_num 54 ranges {163840=540672,704512=376832} 2024-09-16T06:48:24.922 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:24.920+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:24.922 INFO:tasks.rados.rados.0.smithi117.stdout:64: writing smithi11731613-10 from 163840 to 704512 tid 1 2024-09-16T06:48:24.926 INFO:tasks.rados.rados.0.smithi117.stdout:64: writing smithi11731613-10 from 704512 to 1081344 tid 2 2024-09-16T06:48:24.926 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 1 current snap is 2 2024-09-16T06:48:24.926 INFO:tasks.rados.rados.0.smithi117.stdout:65: seq_num 55 ranges {32768=557056,589824=376832} 2024-09-16T06:48:24.931 INFO:tasks.rados.rados.0.smithi117.stdout:65: writing smithi11731613-1 from 32768 to 589824 tid 1 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:65: writing smithi11731613-1 from 589824 to 966656 tid 2 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:66: rollback oid 34 current snap is 2 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 34 to 0 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:59: finishing write tid 1 to smithi11731613-48 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:59: finishing write tid 2 to smithi11731613-48 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:59: finishing write tid 3 to smithi11731613-48 2024-09-16T06:48:24.936 INFO:tasks.rados.rados.0.smithi117.stdout:67: snap_remove snap 1 2024-09-16T06:48:24.937 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:24.937 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:24.975 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:24.975 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:24.983 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:24.983 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:48:25.009 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:25.009 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:25.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:25.008+0000 7f3a157db640 0 -- 172.21.15.100:0/1850301851 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f3a102283a0 msgr2=0x7f3a1022a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:25.023 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:25.024+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:25.121 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:25.121 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.124 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:25.124+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:25.135 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:48:25.224 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:25.223+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:25.242 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:25.303 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:25.303 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:48:25.326 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:25.324+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:25.427 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:25.424+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:25.437 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:25.437 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:48:25.528 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:25.528+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:25.614 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:25.615 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.628 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:25.627+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:25.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:48:25.729 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:25.728+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:25.752 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:25.752 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:48:25.832 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:25.828+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:25.882 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:25.882 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:25.896 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:64: finishing write tid 1 to smithi11731613-10 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:64: finishing write tid 2 to smithi11731613-10 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:59: finishing write tid 4 to smithi11731613-48 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:59: oid 48 updating version 0 to 17 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:59: oid 48 version 17 is already newer than 15 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:59: oid 48 version 17 is already newer than 16 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 48 v 17 (ObjNum 53 snap 2 seq_num 53) dirty exists 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:59: left oid 48 (ObjNum 53 snap 2 seq_num 53) 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:64: finishing write tid 3 to smithi11731613-10 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:64: finishing write tid 4 to smithi11731613-10 2024-09-16T06:48:25.907 INFO:tasks.rados.rados.0.smithi117.stdout:64: oid 10 updating version 0 to 20 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:64: oid 10 version 20 is already newer than 19 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:64: oid 10 updating version 20 to 21 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 21 (ObjNum 54 snap 2 seq_num 54) dirty exists 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:64: left oid 10 (ObjNum 54 snap 2 seq_num 54) 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:66: finishing rollback tid 0 to smithi11731613-34 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:66: finishing rollback tid 1 to smithi11731613-34 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:66: finishing rollback tid 2 to smithi11731613-34 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:59: done (9 left) 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:64: done (8 left) 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:66: done (7 left) 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:67: done (6 left) 2024-09-16T06:48:25.908 INFO:tasks.rados.rados.0.smithi117.stdout:68: snap_create 2024-09-16T06:48:25.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:48:25.931 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:25.929+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:25.960 INFO:tasks.rados.rados.0.smithi117.stdout:63: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-16T06:48:26.001 INFO:tasks.rados.rados.0.smithi117.stdout:62: expect (ObjNum 30 snap 0 seq_num 30) 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:58: finishing rollback tid 0 to smithi11731613-38 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: finishing write tid 1 to smithi11731613-1 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:58: finishing rollback tid 1 to smithi11731613-38 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:58: finishing rollback tid 2 to smithi11731613-38 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 38 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: finishing write tid 2 to smithi11731613-1 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: finishing write tid 3 to smithi11731613-1 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: finishing write tid 4 to smithi11731613-1 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: oid 1 updating version 0 to 9 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: oid 1 version 9 is already newer than 8 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:65: oid 1 updating version 9 to 10 2024-09-16T06:48:26.024 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 1 v 10 (ObjNum 55 snap 2 seq_num 55) dirty exists 2024-09-16T06:48:26.025 INFO:tasks.rados.rados.0.smithi117.stdout:65: left oid 1 (ObjNum 55 snap 2 seq_num 55) 2024-09-16T06:48:26.025 INFO:tasks.rados.rados.0.smithi117.stdout:60: finishing copy_from racing read to smithi11731613-3 2024-09-16T06:48:26.025 INFO:tasks.rados.rados.0.smithi117.stdout:61: finishing rollback tid 0 to smithi11731613-5 2024-09-16T06:48:26.025 INFO:tasks.rados.rados.0.smithi117.stdout:61: finishing rollback tid 1 to smithi11731613-5 2024-09-16T06:48:26.025 INFO:tasks.rados.rados.0.smithi117.stdout:60: finishing copy_from to smithi11731613-3 2024-09-16T06:48:26.025 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-16T06:48:26.031 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:26.027+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:26.066 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.066 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:48:26.132 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:26.132+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:26.195 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.195 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.210 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:48:26.232 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:26.233+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:26.235 INFO:tasks.rados.rados.0.smithi117.stdout:61: finishing rollback tid 2 to smithi11731613-5 2024-09-16T06:48:26.235 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 5 v 14 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T06:48:26.327 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.328 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.333 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:26.332+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:26.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:48:26.434 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:26.432+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:26.465 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.465 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:48:26.535 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:26.531+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:26.603 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.604 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.617 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:48:26.636 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:26.636+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:26.738 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:26.735+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:26.749 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.749 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.763 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:48:26.839 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:26.835+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:26.884 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:26.884 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:26.898 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:48:26.919 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:48:26.920 INFO:tasks.rados.rados.0.smithi117.stdout:58: done (6 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:60: done (5 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:61: done (4 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:62: done (3 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:63: done (2 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:65: done (1 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:68: done (0 left) 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 41 current snap is 3 2024-09-16T06:48:26.921 INFO:tasks.rados.rados.0.smithi117.stdout:69: seq_num 56 ranges {688128=573440,1261568=376832} 2024-09-16T06:48:26.939 INFO:tasks.rados.rados.0.smithi117.stdout:69: writing smithi11731613-41 from 688128 to 1261568 tid 1 2024-09-16T06:48:26.940 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:26.936+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:26.944 INFO:tasks.rados.rados.0.smithi117.stdout:69: writing smithi11731613-41 from 1261568 to 1638400 tid 2 2024-09-16T06:48:26.944 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 23 current snap is 3 2024-09-16T06:48:26.944 INFO:tasks.rados.rados.0.smithi117.stdout:70: seq_num 57 ranges {376832=589824,966656=376832} 2024-09-16T06:48:26.959 INFO:tasks.rados.rados.0.smithi117.stdout:70: writing smithi11731613-23 from 376832 to 966656 tid 1 2024-09-16T06:48:26.965 INFO:tasks.rados.rados.0.smithi117.stdout:70: writing smithi11731613-23 from 966656 to 1343488 tid 2 2024-09-16T06:48:26.965 INFO:tasks.rados.rados.0.smithi117.stdout:71: rollback oid 32 current snap is 3 2024-09-16T06:48:26.965 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 32 to 2 2024-09-16T06:48:26.965 INFO:tasks.rados.rados.0.smithi117.stdout:69: finishing write tid 1 to smithi11731613-41 2024-09-16T06:48:26.966 INFO:tasks.rados.rados.0.smithi117.stdout:69: finishing write tid 2 to smithi11731613-41 2024-09-16T06:48:26.966 INFO:tasks.rados.rados.0.smithi117.stdout:69: finishing write tid 3 to smithi11731613-41 2024-09-16T06:48:26.966 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 26 current snap is 3 2024-09-16T06:48:26.966 INFO:tasks.rados.rados.0.smithi117.stdout:72: seq_num 58 ranges {425984=606208,1032192=376832} 2024-09-16T06:48:26.981 INFO:tasks.rados.rados.0.smithi117.stdout:72: writing smithi11731613-26 from 425984 to 1032192 tid 1 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:72: writing smithi11731613-26 from 1032192 to 1409024 tid 2 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 21 current snap is 3 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:69: finishing write tid 4 to smithi11731613-41 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:69: oid 41 updating version 0 to 18 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:69: oid 41 version 18 is already newer than 16 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:69: oid 41 version 18 is already newer than 17 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 41 v 18 (ObjNum 56 snap 3 seq_num 56) dirty exists 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:69: left oid 41 (ObjNum 56 snap 3 seq_num 56) 2024-09-16T06:48:26.986 INFO:tasks.rados.rados.0.smithi117.stdout:73: seq_num 59 ranges {344064=622592,966656=376832} 2024-09-16T06:48:26.996 INFO:tasks.rados.rados.0.smithi117.stdout:73: writing smithi11731613-21 from 344064 to 966656 tid 1 2024-09-16T06:48:27.000 INFO:tasks.rados.rados.0.smithi117.stdout:73: writing smithi11731613-21 from 966656 to 1343488 tid 2 2024-09-16T06:48:27.000 INFO:tasks.rados.rados.0.smithi117.stdout:69: done (4 left) 2024-09-16T06:48:27.001 INFO:tasks.rados.rados.0.smithi117.stdout:74: delete oid 7 current snap is 3 2024-09-16T06:48:27.001 INFO:tasks.rados.rados.0.smithi117.stdout:71: finishing rollback tid 0 to smithi11731613-32 2024-09-16T06:48:27.001 INFO:tasks.rados.rados.0.smithi117.stdout:71: finishing rollback tid 1 to smithi11731613-32 2024-09-16T06:48:27.001 INFO:tasks.rados.rados.0.smithi117.stdout:71: finishing rollback tid 2 to smithi11731613-32 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 32 v 29 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: finishing write tid 1 to smithi11731613-23 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: finishing write tid 2 to smithi11731613-23 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: finishing write tid 3 to smithi11731613-23 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: finishing write tid 4 to smithi11731613-23 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: oid 23 updating version 0 to 15 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: oid 23 version 15 is already newer than 13 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: oid 23 version 15 is already newer than 14 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 23 v 15 (ObjNum 57 snap 3 seq_num 57) dirty exists 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:70: left oid 23 (ObjNum 57 snap 3 seq_num 57) 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:72: finishing write tid 1 to smithi11731613-26 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:72: finishing write tid 2 to smithi11731613-26 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:72: finishing write tid 3 to smithi11731613-26 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:72: finishing write tid 4 to smithi11731613-26 2024-09-16T06:48:27.002 INFO:tasks.rados.rados.0.smithi117.stdout:72: oid 26 updating version 0 to 7 2024-09-16T06:48:27.003 INFO:tasks.rados.rados.0.smithi117.stdout:72: oid 26 updating version 7 to 8 2024-09-16T06:48:27.003 INFO:tasks.rados.rados.0.smithi117.stdout:72: oid 26 version 8 is already newer than 6 2024-09-16T06:48:27.003 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 26 v 8 (ObjNum 58 snap 3 seq_num 58) dirty exists 2024-09-16T06:48:27.003 INFO:tasks.rados.rados.0.smithi117.stdout:72: left oid 26 (ObjNum 58 snap 3 seq_num 58) 2024-09-16T06:48:27.006 INFO:tasks.rados.rados.0.smithi117.stdout:73: finishing write tid 1 to smithi11731613-21 2024-09-16T06:48:27.008 INFO:tasks.rados.rados.0.smithi117.stdout:73: finishing write tid 2 to smithi11731613-21 2024-09-16T06:48:27.009 INFO:tasks.rados.rados.0.smithi117.stdout:73: finishing write tid 3 to smithi11731613-21 2024-09-16T06:48:27.012 INFO:tasks.rados.rados.0.smithi117.stdout:73: finishing write tid 4 to smithi11731613-21 2024-09-16T06:48:27.012 INFO:tasks.rados.rados.0.smithi117.stdout:73: oid 21 updating version 0 to 20 2024-09-16T06:48:27.012 INFO:tasks.rados.rados.0.smithi117.stdout:73: oid 21 updating version 20 to 21 2024-09-16T06:48:27.012 INFO:tasks.rados.rados.0.smithi117.stdout:73: oid 21 updating version 21 to 22 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 21 v 22 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:73: left oid 21 (ObjNum 59 snap 3 seq_num 59) 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:70: done (4 left) 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:71: done (3 left) 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:72: done (2 left) 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:73: done (1 left) 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:74: done (0 left) 2024-09-16T06:48:27.013 INFO:tasks.rados.rados.0.smithi117.stdout:75: snap_create 2024-09-16T06:48:27.019 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:27.019 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:27.034 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:48:27.040 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:27.040+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:27.142 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:27.140+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:27.155 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:27.155 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:27.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:48:27.243 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:27.244+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:27.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:27.334 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:27.334 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:27.343 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:27.340+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:27.348 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:48:27.426 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:27.427 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:27.444 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:27.444+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:27.449 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:27.449 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:27.473 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:27.473 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:27.477 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:27.478 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:27.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:48:27.506 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:27.506 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:27.530 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:27.531 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:27.545 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:27.544+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:27.556 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:27.556 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:27.585 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:27.606 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:27.606 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:27.616 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:27.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:48:27.644 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:27.644 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:27.646 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:27.644+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:27.672 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:27.704 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:27.736 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:27.736 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:27.743 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:27.743 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:27.744 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:27.744 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:27.747 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:27.743+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:27.757 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:48:27.765 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:27.793 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:27.793 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:27.824 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:27.824 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:27.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:27.824+0000 7f211aafc640 0 -- 172.21.15.100:0/4144675651 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f2114221f80 msgr2=0x7f2114224370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-16T06:48:27.848 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:27.848+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=0, reqid=client.4511.0:1, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.881 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896059+0000", 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9855054049999996, 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034435659999999999, 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896059+0000", 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896059+0000", 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896057+0000", 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896147+0000", 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9357000000000005e-05 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896273+0000", 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012586100000000001 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896297+0000", 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4173999999999999e-05 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896442+0000", 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000145625 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896478+0000", 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5880000000000002e-05 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899437+0000", 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002958792 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899503+0000", 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5959999999999999e-05 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/32 ECSubWrite(tid=0, reqid=client.4511.0:10, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898824+0000", 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9827408520000001, 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003449092, 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.886 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898824+0000", 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898824+0000", 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898823+0000", 2024-09-16T06:48:27.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898899+0000", 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5819e-05 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899029+0000", 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012984300000000001 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899053+0000", 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4400999999999999e-05 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899184+0000", 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000130443 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899205+0000", 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1744999999999999e-05 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902216+0000", 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030107409999999999 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902273+0000", 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6969999999999998e-05 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.890 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9819166199999998, 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064369220000000003, 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899647+0000", 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899689+0000", 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2429000000000003e-05 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899768+0000", 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9277999999999995e-05 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899797+0000", 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8473999999999999e-05 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903489+0000", 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036918680000000001 2024-09-16T06:48:27.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903519+0000", 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0283000000000001e-05 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906024+0000", 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025045430000000001 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906085+0000", 2024-09-16T06:48:27.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1637000000000006e-05 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=2, reqid=client.4511.0:34, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.917931+0000", 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.963634044, 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004345916, 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917931+0000", 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917931+0000", 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917930+0000", 2024-09-16T06:48:27.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918094+0000", 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016369 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918269+0000", 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000174935 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918294+0000", 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5250999999999999e-05 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918470+0000", 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000176081 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918495+0000", 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4828e-05 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922227+0000", 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037319929999999999 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922277+0000", 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9914999999999999e-05 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.899 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=3, reqid=client.4511.0:35, at_version=35'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.918102+0000", 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9634626859999997, 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042176770000000004, 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918102+0000", 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918102+0000", 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918101+0000", 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918128+0000", 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.639e-05 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918343+0000", 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021569499999999999 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918362+0000", 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814999999999999e-05 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919512+0000", 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011494210000000001 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919534+0000", 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2452000000000001e-05 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922287+0000", 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027528869999999999 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922320+0000", 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2863e-05 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.903 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=2, reqid=client.4511.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.919048+0000", 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9625172830000004, 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030523239999999999, 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919048+0000", 2024-09-16T06:48:27.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919048+0000", 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919045+0000", 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919078+0000", 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2820000000000001e-05 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919173+0000", 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4909999999999998e-05 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919197+0000", 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4008000000000001e-05 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919330+0000", 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133289 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919355+0000", 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5041e-05 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922025+0000", 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026696990000000002 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922100+0000", 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4943000000000004e-05 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 35/32 ECSubWrite(tid=1, reqid=client.4511.0:47, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.920711+0000", 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9608543340000004, 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031601490000000001, 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920711+0000", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920711+0000", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920709+0000", 2024-09-16T06:48:27.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920763+0000", 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3838999999999997e-05 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920928+0000", 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016439300000000001 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920944+0000", 2024-09-16T06:48:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6067999999999999e-05 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922187+0000", 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012434690000000001 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922211+0000", 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3354e-05 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923808+0000", 2024-09-16T06:48:27.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015975360000000001 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923871+0000", 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.266e-05 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=7, reqid=client.4511.0:82, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.979333+0000", 2024-09-16T06:48:27.912 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9022317859999998, 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030850650000000001, 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979333+0000", 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979333+0000", 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979331+0000", 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979796+0000", 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00046499200000000001 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979920+0000", 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012379700000000001 2024-09-16T06:48:27.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979942+0000", 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1403000000000002e-05 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.980042+0000", 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100129 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.980060+0000", 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8079000000000001e-05 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982358+0000", 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002297683 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982418+0000", 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0608e-05 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=8, reqid=client.4511.0:124, at_version=35'5, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.035793+0000", 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8457714750000003, 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00290742, 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035793+0000", 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035793+0000", 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035791+0000", 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036213+0000", 2024-09-16T06:48:27.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042232899999999999 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036307+0000", 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3579000000000006e-05 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036338+0000", 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1436999999999997e-05 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036503+0000", 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000165162 2024-09-16T06:48:27.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036529+0000", 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6168000000000001e-05 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.038655+0000", 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021257630000000001 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.038701+0000", 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5723999999999998e-05 2024-09-16T06:48:27.920 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=18, reqid=client.4511.0:130, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8359916209999998, 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044945139999999998, 2024-09-16T06:48:27.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046385+0000", 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00081262499999999998 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046429+0000", 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3606000000000002e-05 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046441+0000", 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2021000000000001e-05 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046556+0000", 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114869 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046571+0000", 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5214e-05 2024-09-16T06:48:27.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050032+0000", 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00346102 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050068+0000", 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5651000000000001e-05 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=19, reqid=client.4511.0:131, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.046631+0000", 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8349340640000005, 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038867540000000001, 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046631+0000", 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046631+0000", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046630+0000", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046655+0000", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4590999999999999e-05 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046704+0000", 2024-09-16T06:48:27.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9693e-05 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046720+0000", 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5415999999999999e-05 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047254+0000", 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00053385699999999999 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047262+0000", 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7569999999999993e-06 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050474+0000", 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032111470000000001 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050518+0000", 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4039000000000003e-05 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.929 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8327495909999998, 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052817840000000003, 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048832+0000", 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7638e-05 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048870+0000", 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8114000000000001e-05 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048877+0000", 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4849999999999997e-06 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.049092+0000", 2024-09-16T06:48:27.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000214806 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.049100+0000", 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8170000000000005e-06 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054030+0000", 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049292750000000003 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054097+0000", 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7453000000000006e-05 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=10, reqid=client.4511.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8303974250000001, 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005244037, 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051166+0000", 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051182+0000", 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5571000000000001e-05 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051230+0000", 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7703000000000001e-05 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051243+0000", 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3573e-05 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051327+0000", 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4098999999999996e-05 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051343+0000", 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5812e-05 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056375+0000", 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050313950000000001 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056411+0000", 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6890000000000001e-05 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=20, reqid=client.4511.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051662+0000", 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8299033229999999, 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047138700000000002, 2024-09-16T06:48:27.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051662+0000", 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051662+0000", 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051660+0000", 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051678+0000", 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7470999999999999e-05 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051735+0000", 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7170000000000003e-05 2024-09-16T06:48:27.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051748+0000", 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2884999999999999e-05 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051827+0000", 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9423000000000002e-05 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051842+0000", 2024-09-16T06:48:27.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4385e-05 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056338+0000", 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044967109999999996 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056375+0000", 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6955000000000001e-05 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=15, reqid=client.4511.0:133, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.052599+0000", 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.828966029, 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00380971, 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052599+0000", 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052599+0000", 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052598+0000", 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052728+0000", 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000129868 2024-09-16T06:48:27.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052781+0000", 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3672999999999999e-05 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052792+0000", 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0312000000000001e-05 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052869+0000", 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7670999999999995e-05 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052895+0000", 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5829999999999998e-05 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056381+0000", 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034860619999999998 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056409+0000", 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7381000000000001e-05 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=16, reqid=client.4511.0:134, at_version=35'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.054246+0000", 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8273190560000003, 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003580537, 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054246+0000", 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054246+0000", 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054245+0000", 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054265+0000", 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9826e-05 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054329+0000", 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4335e-05 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054342+0000", 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3512999999999999e-05 2024-09-16T06:48:27.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054433+0000", 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0084e-05 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054449+0000", 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6393e-05 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057786+0000", 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033369269999999999 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057826+0000", 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0460000000000002e-05 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=22, reqid=client.4511.0:136, at_version=35'3, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.055548+0000", 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8260171999999999, 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032364479999999998, 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.951 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.055548+0000", 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.055548+0000", 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.055547+0000", 2024-09-16T06:48:27.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056626+0000", 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010794000000000001 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056729+0000", 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010320899999999999 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056744+0000", 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4470999999999999e-05 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056895+0000", 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000150824 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056909+0000", 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3871e-05 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.058746+0000", 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018376180000000001 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.058784+0000", 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7926000000000002e-05 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=29, reqid=client.4511.0:159, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:27.955 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.512622+0000", 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3689432850000003, 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047019150000000001, 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.512622+0000", 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.512622+0000", 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.512620+0000", 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513065+0000", 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044465900000000003 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513171+0000", 2024-09-16T06:48:27.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010642 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513195+0000", 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3587e-05 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513371+0000", 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017610999999999999 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513395+0000", 2024-09-16T06:48:27.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4283000000000001e-05 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.517253+0000", 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038581560000000002 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.517324+0000", 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0359999999999997e-05 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.959 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=30, reqid=client.4511.0:160, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.366916067, 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056229460000000002, 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514647+0000", 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515002+0000", 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000354607 2024-09-16T06:48:27.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515104+0000", 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102607 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515128+0000", 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3998e-05 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515678+0000", 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055015800000000003 2024-09-16T06:48:27.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515703+0000", 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4706999999999999e-05 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520193+0000", 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044896160000000001 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520272+0000", 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9089000000000001e-05 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.963 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=31, reqid=client.4511.0:161, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3663870630000003, 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056966849999999999, 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515177+0000", 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515216+0000", 2024-09-16T06:48:27.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9765999999999998e-05 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515642+0000", 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042582399999999999 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515672+0000", 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9403e-05 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:27.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516695+0000", 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010230549999999999 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516716+0000", 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1858e-05 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520824+0000", 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041076059999999998 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:27.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520875+0000", 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0479000000000002e-05 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:27.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:48:27.971 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:27.968+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:27.971 INFO:tasks.rados.rados.0.smithi117.stdout:75: done (0 left) 2024-09-16T06:48:27.971 INFO:tasks.rados.rados.0.smithi117.stdout:76: copy_from oid 42 from oid 17 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:77: read oid 13 snap -1 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:77: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:78: rollback oid 8 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 8 to 2 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:79: rollback oid 4 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 4 to 2 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:80: read oid 15 snap 0 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:80: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:81: copy_from oid 47 from oid 21 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:82: copy_from oid 7 from oid 46 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:83: rollback oid 3 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 3 to 0 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 31 current snap is 4 2024-09-16T06:48:27.972 INFO:tasks.rados.rados.0.smithi117.stdout:84: seq_num 61 ranges {507904=655360,1163264=376832} 2024-09-16T06:48:27.973 INFO:tasks.rados.rados.0.smithi117.stdout:84: writing smithi11731613-31 from 507904 to 1163264 tid 1 2024-09-16T06:48:27.977 INFO:tasks.rados.rados.0.smithi117.stdout:84: writing smithi11731613-31 from 1163264 to 1540096 tid 2 2024-09-16T06:48:27.977 INFO:tasks.rados.rados.0.smithi117.stdout:85: delete oid 14 current snap is 4 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:79: finishing rollback tid 0 to smithi11731613-4 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:83: finishing rollback tid 0 to smithi11731613-3 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:83: finishing rollback tid 1 to smithi11731613-3 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:83: finishing rollback tid 2 to smithi11731613-3 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 14 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:78: finishing rollback tid 0 to smithi11731613-8 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:78: finishing rollback tid 1 to smithi11731613-8 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:79: finishing rollback tid 1 to smithi11731613-4 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:79: finishing rollback tid 2 to smithi11731613-4 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 4 v 14 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T06:48:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:77: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:28.001 INFO:tasks.rados.rados.0.smithi117.stdout:80: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:76: finishing copy_from racing read to smithi11731613-42 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:82: finishing copy_from racing read to smithi11731613-7 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:78: finishing rollback tid 2 to smithi11731613-8 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 8 v 19 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:82: finishing copy_from to smithi11731613-7 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 7 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:28.019 INFO:tasks.rados.rados.0.smithi117.stdout:84: finishing write tid 1 to smithi11731613-31 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:77: done (9 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:78: done (8 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:79: done (7 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:80: done (6 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:82: done (5 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:83: done (4 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:85: done (3 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:86: rmattr oid 22 current snap is 4 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:86: done (3 left) 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 44 current snap is 4 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:84: finishing write tid 2 to smithi11731613-31 2024-09-16T06:48:28.020 INFO:tasks.rados.rados.0.smithi117.stdout:87: seq_num 63 ranges {737280=688128,1425408=376832} 2024-09-16T06:48:28.037 INFO:tasks.rados.rados.0.smithi117.stdout:87: writing smithi11731613-44 from 737280 to 1425408 tid 1 2024-09-16T06:48:28.043 INFO:tasks.rados.rados.0.smithi117.stdout:87: writing smithi11731613-44 from 1425408 to 1802240 tid 2 2024-09-16T06:48:28.043 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 14 current snap is 4 2024-09-16T06:48:28.043 INFO:tasks.rados.rados.0.smithi117.stdout:88: seq_num 64 ranges {0=704512,704512=376832} 2024-09-16T06:48:28.052 INFO:tasks.rados.rados.0.smithi117.stdout:88: writing smithi11731613-14 from 0 to 704512 tid 1 2024-09-16T06:48:28.057 INFO:tasks.rados.rados.0.smithi117.stdout:88: writing smithi11731613-14 from 704512 to 1081344 tid 2 2024-09-16T06:48:28.057 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 48 current snap is 4 2024-09-16T06:48:28.057 INFO:tasks.rados.rados.0.smithi117.stdout:89: seq_num 65 ranges {1703936=720896,2424832=376832} 2024-09-16T06:48:28.066 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:28.066 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:28.070 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:28.068+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:28.080 INFO:tasks.rados.rados.0.smithi117.stdout:89: writing smithi11731613-48 from 1703936 to 2424832 tid 1 2024-09-16T06:48:28.085 INFO:tasks.rados.rados.0.smithi117.stdout:89: writing smithi11731613-48 from 2424832 to 2801664 tid 2 2024-09-16T06:48:28.085 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 3 current snap is 4 2024-09-16T06:48:28.085 INFO:tasks.rados.rados.0.smithi117.stdout:84: finishing write tid 3 to smithi11731613-31 2024-09-16T06:48:28.085 INFO:tasks.rados.rados.0.smithi117.stdout:76: finishing copy_from to smithi11731613-42 2024-09-16T06:48:28.085 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 42 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:84: finishing write tid 4 to smithi11731613-31 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:84: oid 31 updating version 0 to 17 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:84: oid 31 updating version 17 to 18 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:84: oid 31 version 18 is already newer than 16 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 31 v 18 (ObjNum 61 snap 4 seq_num 61) dirty exists 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:84: left oid 31 (ObjNum 61 snap 4 seq_num 61) 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:81: finishing copy_from racing read to smithi11731613-47 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:81: finishing copy_from to smithi11731613-47 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 47 v 22 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:87: finishing write tid 1 to smithi11731613-44 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:87: finishing write tid 2 to smithi11731613-44 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:87: finishing write tid 3 to smithi11731613-44 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:87: finishing write tid 4 to smithi11731613-44 2024-09-16T06:48:28.086 INFO:tasks.rados.rados.0.smithi117.stdout:87: oid 44 updating version 0 to 25 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:87: oid 44 version 25 is already newer than 24 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:87: oid 44 version 25 is already newer than 23 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 25 (ObjNum 63 snap 4 seq_num 63) dirty exists 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:87: left oid 44 (ObjNum 63 snap 4 seq_num 63) 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:88: finishing write tid 1 to smithi11731613-14 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:88: finishing write tid 2 to smithi11731613-14 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:88: finishing write tid 3 to smithi11731613-14 2024-09-16T06:48:28.087 INFO:tasks.rados.rados.0.smithi117.stdout:90: seq_num 66 ranges {49152=737280,786432=376832} 2024-09-16T06:48:28.091 INFO:tasks.rados.rados.0.smithi117.stdout:90: writing smithi11731613-3 from 49152 to 786432 tid 1 2024-09-16T06:48:28.094 INFO:tasks.rados.rados.0.smithi117.stdout:90: writing smithi11731613-3 from 786432 to 1163264 tid 2 2024-09-16T06:48:28.094 INFO:tasks.rados.rados.0.smithi117.stdout:76: done (6 left) 2024-09-16T06:48:28.094 INFO:tasks.rados.rados.0.smithi117.stdout:81: done (5 left) 2024-09-16T06:48:28.095 INFO:tasks.rados.rados.0.smithi117.stdout:84: done (4 left) 2024-09-16T06:48:28.095 INFO:tasks.rados.rados.0.smithi117.stdout:87: done (3 left) 2024-09-16T06:48:28.095 INFO:tasks.rados.rados.0.smithi117.stdout:91: delete oid 12 current snap is 4 2024-09-16T06:48:28.095 INFO:tasks.rados.rados.0.smithi117.stdout:89: finishing write tid 1 to smithi11731613-48 2024-09-16T06:48:28.095 INFO:tasks.rados.rados.0.smithi117.stdout:89: finishing write tid 2 to smithi11731613-48 2024-09-16T06:48:28.095 INFO:tasks.rados.rados.0.smithi117.stdout:89: finishing write tid 3 to smithi11731613-48 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:89: finishing write tid 4 to smithi11731613-48 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:89: oid 48 updating version 0 to 32 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:89: oid 48 updating version 32 to 33 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:89: oid 48 version 33 is already newer than 31 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 48 v 33 (ObjNum 65 snap 4 seq_num 65) dirty exists 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:89: left oid 48 (ObjNum 65 snap 4 seq_num 65) 2024-09-16T06:48:28.099 INFO:tasks.rados.rados.0.smithi117.stdout:90: finishing write tid 1 to smithi11731613-3 2024-09-16T06:48:28.102 INFO:tasks.rados.rados.0.smithi117.stdout:90: finishing write tid 2 to smithi11731613-3 2024-09-16T06:48:28.102 INFO:tasks.rados.rados.0.smithi117.stdout:90: finishing write tid 3 to smithi11731613-3 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:90: finishing write tid 4 to smithi11731613-3 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:90: oid 3 updating version 0 to 15 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:90: oid 3 updating version 15 to 17 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:90: oid 3 version 17 is already newer than 16 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 17 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:90: left oid 3 (ObjNum 66 snap 4 seq_num 66) 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:89: done (3 left) 2024-09-16T06:48:28.105 INFO:tasks.rados.rados.0.smithi117.stdout:90: done (2 left) 2024-09-16T06:48:28.106 INFO:tasks.rados.rados.0.smithi117.stdout:91: done (1 left) 2024-09-16T06:48:28.106 INFO:tasks.rados.rados.0.smithi117.stdout:92: read oid 36 snap 0 2024-09-16T06:48:28.106 INFO:tasks.rados.rados.0.smithi117.stdout:92: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:48:28.106 INFO:tasks.rados.rados.0.smithi117.stdout:93: snap_create 2024-09-16T06:48:28.112 INFO:tasks.rados.rados.0.smithi117.stdout:92: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:48:28.147 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:28.147 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:28.147 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:28.161 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:48:28.172 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:28.167+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:28.273 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:28.271+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:28.282 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:28.282 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:28.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:48:28.374 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:28.371+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:7 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.526237257, 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023918795999999999, 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895875+0000", 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896153+0000", 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000277688 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896194+0000", 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1916000000000003e-05 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896222+0000", 2024-09-16T06:48:28.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.726e-05 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896390+0000", 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016809299999999999 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896750+0000", 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036068999999999999 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898496+0000", 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017459070000000001 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906280+0000", 2024-09-16T06:48:28.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077831310000000004 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919636+0000", 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013356191 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919798+0000", 2024-09-16T06:48:28.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016195000000000001 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:8 3.1s0 3:825f7e25:::smithi11731613-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.525586831, 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031459483000000003, 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.430 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 8 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896527+0000", 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896557+0000", 2024-09-16T06:48:28.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9323999999999999e-05 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896578+0000", 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1038000000000001e-05 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896594+0000", 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6038000000000001e-05 2024-09-16T06:48:28.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900822+0000", 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042283219999999996 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901041+0000", 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021897799999999999 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902173+0000", 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00113226 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911105+0000", 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089313959999999994 2024-09-16T06:48:28.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927760+0000", 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016655313000000001 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927989+0000", 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022873699999999999 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.436 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:9 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5255058019999996, 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.040864080999999997, 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.437 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 9 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:48:28.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896609+0000", 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896617+0000", 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9589999999999995e-06 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896634+0000", 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6872e-05 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896647+0000", 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3621999999999999e-05 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902794+0000", 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061462820000000003 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903015+0000", 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221487 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928840+0000", 2024-09-16T06:48:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025824686999999999 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929046+0000", 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020604299999999999 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.937474+0000", 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084285460000000003 2024-09-16T06:48:28.442 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:16 3.bs0 3:db3fccb6:::smithi11731613-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.525456191, 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024707216000000001, 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.443 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896659+0000", 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897043+0000", 2024-09-16T06:48:28.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038430400000000003 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897068+0000", 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5417000000000001e-05 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897087+0000", 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8218000000000001e-05 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897200+0000", 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011297499999999999 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897502+0000", 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030200600000000001 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899657+0000", 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021558250000000001 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911012+0000", 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011354401 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921200+0000", 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010188033000000001 2024-09-16T06:48:28.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921367+0000", 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016735 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:17 3.bs0 3:db3fccb6:::smithi11731613-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:28.449 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5242291640000003, 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034791753000000002, 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 17 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897885+0000", 2024-09-16T06:48:28.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897900+0000", 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5515e-05 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897935+0000", 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4703000000000003e-05 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897965+0000", 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9884000000000001e-05 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905007+0000", 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007042296 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905241+0000", 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023388 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014152209999999999 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912065+0000", 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054085640000000003 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932510+0000", 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020444922000000001 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932679+0000", 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168894 2024-09-16T06:48:28.455 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:18 3.bs0 3:db3fccb6:::smithi11731613-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5240706270000004, 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.041379560000000003, 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.456 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 18 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898044+0000", 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898055+0000", 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0902e-05 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898077+0000", 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2395000000000002e-05 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898095+0000", 2024-09-16T06:48:28.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7825e-05 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907329+0000", 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092342250000000004 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907544+0000", 2024-09-16T06:48:28.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021447400000000001 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932831+0000", 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025286896999999999 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933015+0000", 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018463 2024-09-16T06:48:28.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939425+0000", 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064098250000000001 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:37 3.bs0 3:d4957fab:::smithi11731613-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:48:28.462 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5169314229999999, 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029935371999999998, 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 37 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:48:28.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905182+0000", 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:28.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905624+0000", 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044199799999999998 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905656+0000", 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2393e-05 2024-09-16T06:48:28.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905676+0000", 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0285e-05 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907630+0000", 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019531459999999998 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907646+0000", 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6628000000000001e-05 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914043+0000", 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063962699999999999 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914333+0000", 2024-09-16T06:48:28.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029089300000000002 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916429+0000", 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002095085 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923501+0000", 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007072404 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934947+0000", 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011445688000000001 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.935120+0000", 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017347200000000001 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.469 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:38 3.bs0 3:d4957fab:::smithi11731613-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.516418936, 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034120408999999997, 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 38 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905696+0000", 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905706+0000", 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9890000000000004e-06 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905725+0000", 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9341e-05 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905739+0000", 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4336e-05 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907725+0000", 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001985849 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907741+0000", 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5401e-05 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918115+0000", 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010374431999999999 2024-09-16T06:48:28.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918349+0000", 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023440900000000001 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919836+0000", 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014862639999999999 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.925093+0000", 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052577300000000004 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939675+0000", 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014581904999999999 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939818+0000", 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014229600000000001 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:39 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5163628080000002, 2024-09-16T06:48:28.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039069524000000001, 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 39 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:48:28.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905752+0000", 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905759+0000", 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905776+0000", 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6694e-05 2024-09-16T06:48:28.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905788+0000", 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2336e-05 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907813+0000", 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020251790000000002 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907828+0000", 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5094e-05 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921873+0000", 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014044235 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922092+0000", 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000219514 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940712+0000", 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018619988000000001 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940891+0000", 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017858199999999999 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.944823+0000", 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039321989999999999 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:40 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5163148929999997, 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027254417, 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 40 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.485 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905800+0000", 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906496+0000", 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00069684800000000002 2024-09-16T06:48:28.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906524+0000", 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7344000000000001e-05 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906543+0000", 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9468000000000001e-05 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906717+0000", 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000173692 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906758+0000", 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1186000000000002e-05 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.913762+0000", 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070039339999999999 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914055+0000", 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029257000000000003 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916068+0000", 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020128730000000001 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921559+0000", 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054913260000000004 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932938+0000", 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011379031 2024-09-16T06:48:28.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933056+0000", 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117785 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:41 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.492 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5155388930000004, 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027013423000000002, 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 41 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906575+0000", 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906603+0000", 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8030000000000001e-05 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906622+0000", 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9008000000000002e-05 2024-09-16T06:48:28.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906638+0000", 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5469e-05 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906985+0000", 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034709699999999999 2024-09-16T06:48:28.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906998+0000", 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3512e-05 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917405+0000", 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010406425 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917725+0000", 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032018999999999998 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919697+0000", 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019721069999999999 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924833+0000", 2024-09-16T06:48:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051359170000000003 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933307+0000", 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084737270000000003 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933591+0000", 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028407699999999998 2024-09-16T06:48:28.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:42 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5154602669999999, 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031605503, 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 42 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906654+0000", 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906661+0000", 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5719999999999999e-06 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906678+0000", 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7059e-05 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906698+0000", 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9551e-05 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907075+0000", 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037732500000000002 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907087+0000", 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1745e-05 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920576+0000", 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013489655 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920843+0000", 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026655899999999998 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934288+0000", 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013445297 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934560+0000", 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000272238 2024-09-16T06:48:28.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.938261+0000", 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037009289999999999 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/32 ECSubWrite(tid=2, reqid=client.4511.0:19, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5152546390000001, 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018027405, 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:28.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907090+0000", 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229569 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919015+0000", 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011924924999999999 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919062+0000", 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6316000000000002e-05 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919376+0000", 2024-09-16T06:48:28.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031417700000000001 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919401+0000", 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4854999999999998e-05 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924838+0000", 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054377970000000003 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924889+0000", 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0445000000000002e-05 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:105 3.6s0 3:64026cd9:::smithi11731613-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4155326969999997, 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023735583000000001, 2024-09-16T06:48:28.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 105 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006587+0000", 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6360000000000002e-06 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006598+0000", 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1151999999999999e-05 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006608+0000", 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0036e-05 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.011103+0000", 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044949100000000004 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.011267+0000", 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000163778 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018815+0000", 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007547705 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018930+0000", 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115093 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.030319+0000", 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011389132999999999 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:108 3.6s0 3:69f2a009:::smithi11731613-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4117909180000003, 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021014327999999999, 2024-09-16T06:48:28.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 108 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010324+0000", 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010330+0000", 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4829999999999996e-06 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010348+0000", 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7878000000000001e-05 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010365+0000", 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6770999999999999e-05 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020964+0000", 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010598849 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021064+0000", 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100537 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028570+0000", 2024-09-16T06:48:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075053480000000002 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028742+0000", 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017269299999999999 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.031340+0000", 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002597168 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.525 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:111 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4083408669999997, 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019128665999999999, 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 111 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:48:28.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013774+0000", 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013781+0000", 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7320000000000001e-06 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013798+0000", 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7665000000000001e-05 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013812+0000", 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3745e-05 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018077+0000", 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004264561 2024-09-16T06:48:28.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018218+0000", 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014137000000000001 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.023743+0000", 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055253430000000003 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.023904+0000", 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016036100000000001 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.032904+0000", 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090001990000000004 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.531 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:113 3.1s0 3:87a9bcdd:::smithi11731613-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4042797140000003, 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017992622999999999, 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 113 2024-09-16T06:48:28.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017835+0000", 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017845+0000", 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9189999999999994e-06 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017876+0000", 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1446999999999999e-05 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017889+0000", 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2571e-05 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020765+0000", 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028765259999999999 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020877+0000", 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111523 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021333+0000", 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00045663300000000002 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.022097+0000", 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00076327800000000005 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035682+0000", 2024-09-16T06:48:28.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013585000999999999 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035829+0000", 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000147367 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:114 3.1s0 3:87a9bcdd:::smithi11731613-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4042159490000001, 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021941939000000001, 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 114 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.539 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017899+0000", 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017905+0000", 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7209999999999999e-06 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017917+0000", 2024-09-16T06:48:28.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1979e-05 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017926+0000", 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1430000000000004e-06 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021590+0000", 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036640029999999999 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021682+0000", 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2441000000000005e-05 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036527+0000", 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014844758 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036729+0000", 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020231099999999999 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.039842+0000", 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031126779999999998 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.544 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:152 3.7s0 3:ebfb55a4:::smithi11731613-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3457087200000002, 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44514418, 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 152 2024-09-16T06:48:28.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076406+0000", 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076414+0000", 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2889999999999998e-06 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076425+0000", 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0671e-05 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076433+0000", 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0870000000000007e-06 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079120+0000", 2024-09-16T06:48:28.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026864800000000002 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079194+0000", 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4874000000000002e-05 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079523+0000", 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032904499999999998 2024-09-16T06:48:28.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080194+0000", 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067098400000000001 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521344+0000", 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.441149183 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521552+0000", 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020799600000000001 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:153 3.7s0 3:ebfb55a4:::smithi11731613-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3456748940000001, 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44943085999999999, 2024-09-16T06:48:28.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 153 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076440+0000", 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:28.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076444+0000", 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0559999999999998e-06 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076455+0000", 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0789e-05 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076466+0000", 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0241000000000001e-05 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080491+0000", 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040256060000000001 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080588+0000", 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6849999999999996e-05 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:28.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522277+0000", 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44168930299999998 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522519+0000", 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241246 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.525872+0000", 2024-09-16T06:48:28.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033535660000000001 2024-09-16T06:48:28.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.557 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:28.557 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:28.557 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:28.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:48:28.559 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:28.555+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:28.659 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:28.656+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:28.688 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:28.688 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:28.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:48:28.760 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:28.760+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:28.824 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:28.825 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:28.825 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:28.825 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:28.825 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:28.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:48:28.861 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:28.864+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:28.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:92: done (2 left) 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:93: done (1 left) 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:94: read oid 29 snap 4 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:94: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:95: read oid 9 snap -1 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:95: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-16T06:48:28.952 INFO:tasks.rados.rados.0.smithi117.stdout:96: snap_create 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/32 ECSubWrite(tid=0, reqid=client.4511.0:10, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.895427+0000", 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0616268670000002, 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080271079999999998, 2024-09-16T06:48:28.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895427+0000", 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895427+0000", 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895426+0000", 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895563+0000", 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000137069 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895732+0000", 2024-09-16T06:48:28.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000168806 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895755+0000", 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3102999999999999e-05 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895893+0000", 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000137682 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895923+0000", 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0700000000000001e-05 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903386+0000", 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074627720000000003 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903454+0000", 2024-09-16T06:48:28.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7775999999999996e-05 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/32 ECSubWrite(tid=0, reqid=client.4511.0:13, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.896576+0000", 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0604779320000004, 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079004959999999999, 2024-09-16T06:48:28.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896576+0000", 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896576+0000", 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896575+0000", 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896690+0000", 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115072 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896781+0000", 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0759000000000003e-05 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896804+0000", 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-05 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896926+0000", 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012238200000000001 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896948+0000", 2024-09-16T06:48:28.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2147e-05 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904410+0000", 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007462298 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904476+0000", 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5823999999999999e-05 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:14, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.897840+0000", 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0592139310000004, 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066815709999999999, 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897840+0000", 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897840+0000", 2024-09-16T06:48:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897838+0000", 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897882+0000", 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3758999999999999e-05 2024-09-16T06:48:28.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897972+0000", 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9926999999999995e-05 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897994+0000", 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1939000000000001e-05 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.900454+0000", 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002460439 2024-09-16T06:48:28.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.900479+0000", 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5313e-05 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904488+0000", 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040086280000000002 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904521+0000", 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3216000000000002e-05 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs4 35/32 ECSubWrite(tid=4, reqid=client.4511.0:25, at_version=35'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.913460+0000", 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0435932130000003, 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070523249999999999, 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913460+0000", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913460+0000", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913458+0000", 2024-09-16T06:48:28.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913602+0000", 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014330899999999999 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913693+0000", 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1745999999999999e-05 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913716+0000", 2024-09-16T06:48:28.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2359999999999999e-05 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914126+0000", 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041001399999999999 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914150+0000", 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4275999999999999e-05 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.920446+0000", 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062962469999999996 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.920513+0000", 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6422999999999997e-05 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=0, reqid=client.4511.0:46, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.916714+0000", 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0403397559999998, 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073089720000000004, 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916714+0000", 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916714+0000", 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916713+0000", 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917157+0000", 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044355300000000001 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917278+0000", 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121629 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917323+0000", 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4786000000000001e-05 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917483+0000", 2024-09-16T06:48:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015960900000000001 2024-09-16T06:48:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917506+0000", 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3133999999999999e-05 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923869+0000", 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063627989999999997 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924023+0000", 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015419899999999999 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=1, reqid=client.4511.0:47, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917167+0000", 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0398867870000004, 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069026859999999999, 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917167+0000", 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917167+0000", 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917165+0000", 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917196+0000", 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.078e-05 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917376+0000", 2024-09-16T06:48:28.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000179996 2024-09-16T06:48:28.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917423+0000", 2024-09-16T06:48:28.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7219000000000003e-05 2024-09-16T06:48:28.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918706+0000", 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012825880000000001 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918731+0000", 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4939999999999998e-05 2024-09-16T06:48:28.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924033+0000", 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053027170000000002 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924070+0000", 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6213e-05 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/32 ECSubWrite(tid=2, reqid=client.4511.0:34, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.921371+0000", 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0356827839999996, 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066442439999999997, 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921371+0000", 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921371+0000", 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921370+0000", 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921580+0000", 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020980200000000001 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921833+0000", 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025357800000000002 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921852+0000", 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8467000000000001e-05 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922018+0000", 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016620200000000001 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922042+0000", 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3649e-05 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927915+0000", 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058736040000000002 2024-09-16T06:48:28.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928015+0000", 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9949000000000001e-05 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=5, reqid=client.4511.0:59, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0158994579999998, 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063718670000000005, 2024-09-16T06:48:28.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941153+0000", 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941187+0000", 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3837000000000001e-05 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002113+0000", 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060926368000000002 2024-09-16T06:48:28.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002134+0000", 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-05 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002230+0000", 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5789000000000001e-05 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:28.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002246+0000", 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5676e-05 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004821+0000", 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002575262 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004873+0000", 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2095000000000002e-05 2024-09-16T06:48:28.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=158, reqid=client.4511.0:77, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.98150242, 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051524863999999997, 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:28.999 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975550+0000", 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975589+0000", 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9523000000000001e-05 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021558+0000", 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045968459000000003 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021574+0000", 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6399999999999999e-05 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021656+0000", 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1742000000000004e-05 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021673+0000", 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7343e-05 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027012+0000", 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053383889999999998 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027076+0000", 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4325999999999994e-05 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.9s1 35/32 ECSubWrite(tid=7, reqid=client.4511.0:101, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.003754+0000", 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9532997910000001, 2024-09-16T06:48:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067825439999999997, 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.003754+0000", 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.003754+0000", 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.003753+0000", 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.003780+0000", 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6982999999999999e-05 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.005336+0000", 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00155605 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.005346+0000", 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0056e-05 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.005594+0000", 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024791400000000001 2024-09-16T06:48:29.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.005614+0000", 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0251999999999999e-05 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.010507+0000", 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048934800000000004 2024-09-16T06:48:29.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.010536+0000", 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9023e-05 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.009 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.9s1 35/32 ECSubRead(tid=11, to_read={3:92c79891:::smithi11731613-34:head=0,4096,0}, subchunks={3:92c79891:::smithi11731613-34:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:48:29.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:48:29.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.945690903, 2024-09-16T06:48:29.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034091585000000001, 2024-09-16T06:48:29.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:48:29.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011362+0000", 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011374+0000", 2024-09-16T06:48:29.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2490000000000001e-05 2024-09-16T06:48:29.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044980+0000", 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033605818000000003 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044996+0000", 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5996999999999999e-05 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045095+0000", 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9290000000000007e-05 2024-09-16T06:48:29.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045454+0000", 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000359038 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9114624579999999, 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623068000000002, 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045590+0000", 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045617+0000", 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.724e-05 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073137+0000", 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027519538999999999 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073150+0000", 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3046e-05 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073218+0000", 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7879999999999994e-05 2024-09-16T06:48:29.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073228+0000", 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074167+0000", 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093834699999999997 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074214+0000", 2024-09-16T06:48:29.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7722000000000002e-05 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.0s3 35/32 ECSubRead(tid=19, to_read={3:043b7e8c:::smithi11731613-16:head=0,65536,0}, subchunks={3:043b7e8c:::smithi11731613-16:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.8753086919999999, 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.14017474999999999, 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081744+0000", 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081755+0000", 2024-09-16T06:48:29.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0812000000000001e-05 2024-09-16T06:48:29.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221448+0000", 2024-09-16T06:48:29.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.13969311000000001 2024-09-16T06:48:29.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221465+0000", 2024-09-16T06:48:29.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7030000000000001e-05 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221629+0000", 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163549 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221920+0000", 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000291181 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s6 40/37 ECSubWrite(tid=32, reqid=client.4511.0:237, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.958646+0000", 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.99840790800000001, 2024-09-16T06:48:29.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073937589999999997, 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958646+0000", 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958646+0000", 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958645+0000", 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958719+0000", 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3256000000000005e-05 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958889+0000", 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017047199999999999 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958912+0000", 2024-09-16T06:48:29.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2612999999999998e-05 2024-09-16T06:48:29.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.959060+0000", 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014871599999999999 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.959087+0000", 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.656e-05 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.965992+0000", 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069051349999999997 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966040+0000", 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7477e-05 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/37 ECSubWrite(tid=33, reqid=client.4511.0:227, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.958725+0000", 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.99832852500000002, 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072568160000000001, 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958725+0000", 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958725+0000", 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958725+0000", 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958767+0000", 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2126000000000003e-05 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958962+0000", 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019505299999999999 2024-09-16T06:48:29.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.958982+0000", 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0536000000000001e-05 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.959236+0000", 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025372799999999999 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.959260+0000", 2024-09-16T06:48:29.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4267e-05 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.965913+0000", 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066529609999999998 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.965982+0000", 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8708999999999996e-05 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.039 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/37 ECSubWrite(tid=34, reqid=client.4511.0:228, at_version=40'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.962038+0000", 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.99501614699999996, 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007759157, 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962038+0000", 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962038+0000", 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962036+0000", 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962085+0000", 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9039999999999998e-05 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962356+0000", 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027114800000000003 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962378+0000", 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1990000000000001e-05 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962534+0000", 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015646599999999999 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962559+0000", 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5062000000000002e-05 2024-09-16T06:48:29.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.969759+0000", 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071991939999999999 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.969797+0000", 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8087000000000003e-05 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/37 ECSubWrite(tid=36, reqid=client.4511.0:229, at_version=40'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.962177+0000", 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.99487703400000005, 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076532299999999996, 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962177+0000", 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962177+0000", 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962176+0000", 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962214+0000", 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7725999999999998e-05 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962488+0000", 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027467799999999999 2024-09-16T06:48:29.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962503+0000", 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4474000000000001e-05 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.963993+0000", 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001490379 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.964016+0000", 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2429000000000002e-05 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.969802+0000", 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057867839999999997 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.969830+0000", 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.743e-05 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs4 40/37 ECSubWrite(tid=57, reqid=client.4511.0:235, at_version=40'25, trim_to=0'0, roll_forward_to=39'24))", 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.974248+0000", 2024-09-16T06:48:29.050 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.98280580299999998, 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082710379999999997, 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974248+0000", 2024-09-16T06:48:29.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974248+0000", 2024-09-16T06:48:29.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974246+0000", 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974346+0000", 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9705999999999997e-05 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974431+0000", 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4899000000000001e-05 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974453+0000", 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2747000000000001e-05 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974584+0000", 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013027500000000001 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974608+0000", 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.427e-05 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.982455+0000", 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078471080000000002 2024-09-16T06:48:29.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.982519+0000", 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3903000000000001e-05 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/37 ECSubWrite(tid=40, reqid=client.4511.0:240, at_version=40'16, trim_to=0'0, roll_forward_to=40'14))", 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.975288+0000", 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.98176585500000002, 2024-09-16T06:48:29.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090935900000000004, 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975288+0000", 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975288+0000", 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975286+0000", 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976063+0000", 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00077748100000000001 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976183+0000", 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119245 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976199+0000", 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6067e-05 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976306+0000", 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106873 2024-09-16T06:48:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976325+0000", 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9542999999999999e-05 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.984310+0000", 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079843910000000004 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.984381+0000", 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1915999999999994e-05 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.3s5 40/32 ECSubWrite(tid=27, reqid=client.4511.0:226, at_version=40'19, trim_to=0'0, roll_forward_to=40'18))", 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.975738+0000", 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.98131585399999999, 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006871439, 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975738+0000", 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975738+0000", 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975736+0000", 2024-09-16T06:48:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975827+0000", 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1056000000000004e-05 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976108+0000", 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281141 2024-09-16T06:48:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976133+0000", 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4596999999999998e-05 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976279+0000", 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014571500000000001 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976303+0000", 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4037e-05 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.982561+0000", 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062586430000000004 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.982609+0000", 2024-09-16T06:48:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7917000000000002e-05 2024-09-16T06:48:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.066 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:29.067 INFO:tasks.rados.rados.0.smithi117.stdout:95: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-16T06:48:29.068 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:29.064+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:29.068 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:48:29.085 INFO:tasks.rados.rados.0.smithi117.stdout:88: finishing write tid 4 to smithi11731613-14 2024-09-16T06:48:29.085 INFO:tasks.rados.rados.0.smithi117.stdout:88: oid 14 updating version 0 to 15 2024-09-16T06:48:29.085 INFO:tasks.rados.rados.0.smithi117.stdout:88: oid 14 version 15 is already newer than 13 2024-09-16T06:48:29.085 INFO:tasks.rados.rados.0.smithi117.stdout:88: oid 14 version 15 is already newer than 14 2024-09-16T06:48:29.085 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 14 v 15 (ObjNum 64 snap 4 seq_num 64) dirty exists 2024-09-16T06:48:29.086 INFO:tasks.rados.rados.0.smithi117.stdout:88: left oid 14 (ObjNum 64 snap 4 seq_num 64) 2024-09-16T06:48:29.168 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:29.168+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:29.196 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:29.196 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:29.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:48:29.255 INFO:tasks.rados.rados.0.smithi117.stdout:94: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-16T06:48:29.269 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:29.268+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:29.326 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:29.327 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:29.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:48:29.370 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:29.368+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:29.470 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:29.470 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:29.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:29.470 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:29.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.816118008, 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:48:29.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:48:29.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:48:29.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.477 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.184465872000001, 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017661530000000002, 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.478 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282228+0000", 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282240+0000", 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2339e-05 2024-09-16T06:48:29.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282259+0000", 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8499e-05 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282274+0000", 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5673000000000001e-05 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297469+0000", 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015194177999999999 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297683+0000", 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021437800000000001 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297867+0000", 2024-09-16T06:48:29.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018445199999999999 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299716+0000", 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018485590000000001 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299769+0000", 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3192999999999997e-05 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299836+0000", 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7255000000000006e-05 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299891+0000", 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4243999999999998e-05 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.184405192, 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018410340000000001, 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282289+0000", 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282302+0000", 2024-09-16T06:48:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3049e-05 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282319+0000", 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7379000000000001e-05 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282336+0000", 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6881e-05 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.298971+0000", 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016634903999999999 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299117+0000", 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014632600000000001 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299317+0000", 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019987600000000001 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300036+0000", 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071873499999999997 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300588+0000", 2024-09-16T06:48:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055181800000000004 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300655+0000", 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7742000000000003e-05 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300700+0000", 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999998e-05 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.002317829999999, 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:48:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:48:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.002189277999999, 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:48:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:48:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:48:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.002151950999998, 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:48:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:48:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.002016952999998, 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.001984791999998, 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:48:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:48:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:48:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:48:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.001956361000001, 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:48:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:48:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.001849442000001, 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:48:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:48:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.550331462999999, 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018030338, 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 165 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916363+0000", 2024-09-16T06:48:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916367+0000", 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4790000000000001e-06 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916371+0000", 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3460000000000004e-06 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916374+0000", 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4400000000000001e-06 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924029+0000", 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076542229999999999 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924097+0000", 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8849000000000004e-05 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924193+0000", 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5544999999999994e-05 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927143+0000", 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029495250000000002 2024-09-16T06:48:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934287+0000", 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071441270000000001 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934346+0000", 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9521000000000002e-05 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934394+0000", 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-05 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.308342930999999, 2024-09-16T06:48:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:48:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:48:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.308312683, 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:48:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.308283247999999, 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:48:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.308172329000001, 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:48:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:48:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:77 3.8s0 3:119b2380:::smithi11731613-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4950028030000002, 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055956466000000003, 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 77 2024-09-16T06:48:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971691+0000", 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971701+0000", 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8519999999999999e-06 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971715+0000", 2024-09-16T06:48:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4386000000000001e-05 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971725+0000", 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0360000000000001e-05 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.974948+0000", 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003222759 2024-09-16T06:48:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975027+0000", 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8497000000000004e-05 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975520+0000", 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049320500000000003 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.977378+0000", 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018576230000000001 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027491+0000", 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050112973999999998 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027649+0000", 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015816800000000001 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:78 3.8s0 3:119b2380:::smithi11731613-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.494907789, 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059293289999999998, 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971786+0000", 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971793+0000", 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-16T06:48:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971809+0000", 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5225000000000001e-05 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971820+0000", 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0922e-05 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975891+0000", 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040709759999999996 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.976067+0000", 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017645900000000001 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027711+0000", 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051643840000000003 2024-09-16T06:48:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027855+0000", 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014441399999999999 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031081+0000", 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032252359999999998 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 35/32 ECSubWriteReply(tid=162, last_complete=35'4, committed=1, applied=1))", 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4339423260000004, 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48350030599999999, 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032752+0000", 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032763+0000", 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1807000000000001e-05 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516055+0000", 2024-09-16T06:48:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48329140799999998 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516068+0000", 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2782999999999999e-05 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516198+0000", 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129949 2024-09-16T06:48:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516253+0000", 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5390000000000003e-05 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=17, reqid=client.4511.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:48:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4108460620000001, 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.51123769299999999, 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055848+0000", 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055863+0000", 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.464e-05 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565781+0000", 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50991880999999994 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565796+0000", 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4263e-05 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565932+0000", 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013609900000000001 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565949+0000", 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7229000000000001e-05 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567019+0000", 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001070463 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567087+0000", 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7229000000000003e-05 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.7s2 40/37 ECSubRead(tid=11, to_read={3:e60b6e1b:::smithi11731613-14:head=0,4096,0}, subchunks={3:e60b6e1b:::smithi11731613-14:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.399164667, 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.375282063, 2024-09-16T06:48:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067529+0000", 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067547+0000", 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8187000000000001e-05 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442125+0000", 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.37457846099999997 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442142+0000", 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6843000000000001e-05 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442313+0000", 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017038200000000001 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442812+0000", 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049983999999999998 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:29.604 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:29.605 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:29.468+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:29.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:48:29.606 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:29.608+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:29.706 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:29.704+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:29.733 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:29.734 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:29.749 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:48:29.807 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:29.808+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:29.872 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:29.872 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:29.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:48:29.928 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:29.928 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:29.928 INFO:tasks.thrashosds.thrasher:inject_pause on osd.12 2024-09-16T06:48:29.928 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T06:48:29.929 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T06:48:29.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok config set filestore_inject_stall 3 2024-09-16T06:48:29.933 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:29.931+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:29.963 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:88: done (3 left) 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:94: done (2 left) 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:95: done (1 left) 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:96: done (0 left) 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:97: rmattr oid 40 current snap is 6 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:97: done (0 left) 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:98: read oid 49 snap -1 2024-09-16T06:48:29.964 INFO:tasks.rados.rados.0.smithi117.stdout:98: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-16T06:48:29.965 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 41 current snap is 6 2024-09-16T06:48:29.965 INFO:tasks.rados.rados.0.smithi117.stdout:99: seq_num 68 ranges {1638400=770048,2408448=376832} 2024-09-16T06:48:29.997 INFO:tasks.rados.rados.0.smithi117.stdout:99: writing smithi11731613-41 from 1638400 to 2408448 tid 1 2024-09-16T06:48:30.003 INFO:tasks.rados.rados.0.smithi117.stdout:99: writing smithi11731613-41 from 2408448 to 2785280 tid 2 2024-09-16T06:48:30.003 INFO:tasks.rados.rados.0.smithi117.stdout:100: read oid 19 snap -1 2024-09-16T06:48:30.003 INFO:tasks.rados.rados.0.smithi117.stdout:100: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-16T06:48:30.003 INFO:tasks.rados.rados.0.smithi117.stdout:101: read oid 28 snap 3 2024-09-16T06:48:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:101: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-16T06:48:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:102: delete oid 6 current snap is 6 2024-09-16T06:48:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:98: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1215685930000001, 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022146409999999998, 2024-09-16T06:48:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897264+0000", 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897309+0000", 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5123000000000001e-05 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913004+0000", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015694784999999999 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913046+0000", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2945999999999998e-05 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914353+0000", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306104 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914377+0000", 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4562999999999998e-05 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919353+0000", 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049759360000000002 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919412+0000", 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9327000000000003e-05 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:22 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1192930499999996, 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023053033000000001, 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 22 2024-09-16T06:48:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899530+0000", 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999881 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901021+0000", 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014909019999999999 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901056+0000", 2024-09-16T06:48:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5136999999999999e-05 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901077+0000", 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1622999999999999e-05 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901274+0000", 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019618500000000001 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901606+0000", 2024-09-16T06:48:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033268099999999998 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903794+0000", 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021876339999999999 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907586+0000", 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003792489 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922327+0000", 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014741124 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922595+0000", 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026710500000000001 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:23 3.3s0 3:ce1274e9:::smithi11731613-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1177298479999997, 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022719541999999999, 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901103+0000", 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901115+0000", 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1722999999999999e-05 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901134+0000", 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9213000000000001e-05 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901149+0000", 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.523e-05 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905252+0000", 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041033170000000004 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905520+0000", 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026748400000000002 2024-09-16T06:48:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.906744+0000", 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012242209999999999 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923321+0000", 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016576822000000001 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923623+0000", 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249399999999999 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923824+0000", 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000200955 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:24 3.3s0 3:ce1274e9:::smithi11731613-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:48:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1176710219999997, 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033126111999999999, 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901162+0000", 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901169+0000", 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901187+0000", 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8320000000000001e-05 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901203+0000", 2024-09-16T06:48:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5872999999999999e-05 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907294+0000", 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006090949 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907512+0000", 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021778499999999999 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924007+0000", 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016495668000000002 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924218+0000", 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021115199999999999 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934290+0000", 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010071302000000001 2024-09-16T06:48:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:31 3.3s0 3:c7654c4c:::smithi11731613-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1176186990000003, 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026931531000000002, 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31 2024-09-16T06:48:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901215+0000", 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902241+0000", 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010258190000000001 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902275+0000", 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4037e-05 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902312+0000", 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.718e-05 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053663829999999997 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.908030+0000", 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035200800000000001 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912004+0000", 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039736770000000001 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923909+0000", 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011905429 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927953+0000", 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040441499999999998 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928147+0000", 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194005 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:32 3.3s0 3:c7654c4c:::smithi11731613-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1165026410000003, 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028234555000000001, 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 32 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902330+0000", 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902341+0000", 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1279e-05 2024-09-16T06:48:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902360+0000", 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902374+0000", 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3917000000000001e-05 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917230+0000", 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014855607 2024-09-16T06:48:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917486+0000", 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025648799999999997 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918644+0000", 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011572749999999999 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928363+0000", 2024-09-16T06:48:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097195389999999993 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930283+0000", 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919505 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930566+0000", 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028369099999999998 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:33 3.3s0 3:c7654c4c:::smithi11731613-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1164479509999996, 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037765197, 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 33 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902386+0000", 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902392+0000", 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893999999999998e-05 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902429+0000", 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8689999999999999e-05 2024-09-16T06:48:30.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934639+0000", 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032210404999999998 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934843+0000", 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020376199999999999 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.940152+0000", 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053090200000000002 2024-09-16T06:48:30.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 34/32 ECSubWrite(tid=3, reqid=client.4511.0:17, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.903724+0000", 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1151101409999997, 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019548528999999999, 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903724+0000", 2024-09-16T06:48:30.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903724+0000", 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903722+0000", 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903764+0000", 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1103999999999998e-05 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904106+0000", 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034275700000000001 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904129+0000", 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2819999999999998e-05 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904806+0000", 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067673099999999997 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904829+0000", 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.321e-05 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922886+0000", 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018056961 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923273+0000", 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038695 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 35/32 ECSubWriteReply(tid=0, last_complete=35'1, committed=1, applied=1))", 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.907666+0000", 2024-09-16T06:48:30.062 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.111168084, 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014058074, 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907666+0000", 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907666+0000", 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907665+0000", 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2952999999999999e-05 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907789+0000", 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111186 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907809+0000", 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9726999999999999e-05 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921668+0000", 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013859541 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921725+0000", 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-05 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=4, reqid=client.4511.0:44, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.1015312650000002, 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788766000000001, 2024-09-16T06:48:30.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917302+0000", 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917349+0000", 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7796000000000001e-05 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941504+0000", 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024154637 2024-09-16T06:48:30.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941525+0000", 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0608999999999999e-05 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941640+0000", 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011531000000000001 2024-09-16T06:48:30.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941662+0000", 2024-09-16T06:48:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2328999999999999e-05 2024-09-16T06:48:30.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943041+0000", 2024-09-16T06:48:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001379047 2024-09-16T06:48:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943092+0000", 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0834000000000002e-05 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 35/32 ECSubWrite(tid=7, reqid=client.4511.0:38, at_version=35'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.918868+0000", 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.0999670259999998, 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016499218, 2024-09-16T06:48:30.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918868+0000", 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918868+0000", 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918867+0000", 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918941+0000", 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4241999999999998e-05 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919264+0000", 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032274799999999998 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919281+0000", 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7173999999999999e-05 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927297+0000", 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080156040000000008 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927319+0000", 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2566e-05 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.935340+0000", 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080204109999999999 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.935367+0000", 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7246e-05 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:55 3.3s0 3:c48114e8:::smithi11731613-19:head [append 0~311296,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.932003+0000", 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.08683131, 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014166902, 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 55 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.932003+0000", 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.932003+0000", 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.932000+0000", 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.933744+0000", 2024-09-16T06:48:30.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001743353 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.933771+0000", 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6752999999999999e-05 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.933790+0000", 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9570000000000001e-05 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.935897+0000", 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002106611 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.936133+0000", 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023580500000000001 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.937740+0000", 2024-09-16T06:48:30.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607582 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.939612+0000", 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001871412 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.945962+0000", 2024-09-16T06:48:30.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063502680000000001 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.946170+0000", 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020832399999999999 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:68 3.3s0 3:cbbd057c:::smithi11731613-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.064809651, 2024-09-16T06:48:30.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045964098000000002, 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 68 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954024+0000", 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954031+0000", 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954046+0000", 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954058+0000", 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1938e-05 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956544+0000", 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024857949999999998 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956742+0000", 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019819 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.957650+0000", 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090843 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959725+0000", 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074256 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999914+0000", 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040189895000000003 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999989+0000", 2024-09-16T06:48:30.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4527e-05 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:69 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.0647651659999999, 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048288992000000003, 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 69 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954068+0000", 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954074+0000", 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954088+0000", 2024-09-16T06:48:30.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4866e-05 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954100+0000", 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1328999999999999e-05 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959103+0000", 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035749999999997 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959283+0000", 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017953899999999999 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000081+0000", 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040797893000000002 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000261+0000", 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180304 2024-09-16T06:48:30.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002358+0000", 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002097392 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:98 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.0233413000000002, 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.086611296000000004, 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995492+0000", 2024-09-16T06:48:30.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995501+0000", 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8720000000000008e-06 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995517+0000", 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6650000000000002e-05 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995529+0000", 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1447999999999999e-05 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.997874+0000", 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023456250000000001 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998037+0000", 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016212600000000001 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998796+0000", 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075897099999999999 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002377+0000", 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00358195 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081972+0000", 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.079594867 2024-09-16T06:48:30.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082105+0000", 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013220899999999999 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:99 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.0232742369999999, 2024-09-16T06:48:30.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.087893389000000002, 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995559+0000", 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995565+0000", 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2659999999999998e-06 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995581+0000", 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.611e-05 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995594+0000", 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2269000000000001e-05 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999116+0000", 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003522238 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999227+0000", 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111483 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082346+0000", 2024-09-16T06:48:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.083118743999999994 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082416+0000", 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9993000000000006e-05 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.083454+0000", 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00103751 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:158 3.3s0 3:c7654c4c:::smithi11731613-11:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.9390484270000004, 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41982487699999999, 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.106 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079784+0000", 2024-09-16T06:48:30.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079794+0000", 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6260000000000007e-06 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079810+0000", 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6631000000000001e-05 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079821+0000", 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0849000000000001e-05 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079917+0000", 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5730999999999995e-05 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.080126+0000", 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020895399999999999 2024-09-16T06:48:30.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081069+0000", 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094364700000000004 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082604+0000", 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001534863 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499525+0000", 2024-09-16T06:48:30.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41692029400000002 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499611+0000", 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-05 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:224 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955483+0000", 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0633518949999998, 2024-09-16T06:48:30.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016184384, 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 224 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955483+0000", 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955483+0000", 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955480+0000", 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955495+0000", 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5526e-05 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955531+0000", 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.578e-05 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955554+0000", 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3326999999999999e-05 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955686+0000", 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013162199999999999 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955707+0000", 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1077000000000002e-05 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.960259+0000", 2024-09-16T06:48:30.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045520220000000002 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.960499+0000", 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000240215 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.962400+0000", 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001901266 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.968380+0000", 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059793329999999999 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971466+0000", 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003085764 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971667+0000", 2024-09-16T06:48:30.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020151500000000001 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:225 3.3s0 3:ce1274e9:::smithi11731613-8:head [rollback 5] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955580+0000", 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0632546019999998, 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016673753999999999, 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 225 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955580+0000", 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955580+0000", 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955579+0000", 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955585+0000", 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7970000000000003e-06 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955605+0000", 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9023000000000001e-05 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955619+0000", 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4116e-05 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955909+0000", 2024-09-16T06:48:30.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029064100000000001 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955928+0000", 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8340000000000001e-05 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.964517+0000", 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085889639999999993 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.964737+0000", 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022061800000000001 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.967630+0000", 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028932060000000002 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.969873+0000", 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00224274 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.972104+0000", 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022310070000000001 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.972254+0000", 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000149579 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:226 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:30.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0631676259999998, 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027980046000000001, 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 226 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955665+0000", 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955672+0000", 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000002e-06 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955690+0000", 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8179999999999999e-05 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955704+0000", 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.396e-05 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956020+0000", 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000316101 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956038+0000", 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8425999999999999e-05 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974108+0000", 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018069776999999999 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974338+0000", 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022928799999999999 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975586+0000", 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012484639999999999 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.979590+0000", 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040043939999999997 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983481+0000", 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038905480000000002 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983647+0000", 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166045 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.132 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:30.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:30.134 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:30.036+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:30.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:48:30.136 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:30.131+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:30.137 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:30.137 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T06:48:30.137 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:101: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:100: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:99: finishing write tid 1 to smithi11731613-41 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:99: finishing write tid 2 to smithi11731613-41 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:98: done (4 left) 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:100: done (3 left) 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:101: done (2 left) 2024-09-16T06:48:30.139 INFO:tasks.rados.rados.0.smithi117.stdout:102: done (1 left) 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:103: rollback oid 4 current snap is 6 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 4 to 3 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: finishing write tid 3 to smithi11731613-41 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: finishing write tid 4 to smithi11731613-41 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: oid 41 updating version 0 to 29 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: oid 41 version 29 is already newer than 28 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: oid 41 version 29 is already newer than 27 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 41 v 29 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: left oid 41 (ObjNum 68 snap 6 seq_num 68) 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:99: done (1 left) 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:104: rollback oid 43 current snap is 6 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 43 to 4 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:105: read oid 49 snap -1 2024-09-16T06:48:30.140 INFO:tasks.rados.rados.0.smithi117.stdout:105: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:106: copy_from oid 8 from oid 13 current snap is 6 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 39 current snap is 6 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:107: seq_num 70 ranges {638976=802816,1441792=376832} 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:107: writing smithi11731613-39 from 638976 to 1441792 tid 1 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:107: writing smithi11731613-39 from 1441792 to 1818624 tid 2 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:108: copy_from oid 30 from oid 2 current snap is 6 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:104: finishing rollback tid 0 to smithi11731613-43 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:104: finishing rollback tid 1 to smithi11731613-43 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:103: finishing rollback tid 0 to smithi11731613-4 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:103: finishing rollback tid 1 to smithi11731613-4 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:104: finishing rollback tid 2 to smithi11731613-43 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 43 v 11 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-16T06:48:30.141 INFO:tasks.rados.rados.0.smithi117.stdout:103: finishing rollback tid 2 to smithi11731613-4 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 4 v 24 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:106: finishing copy_from racing read to smithi11731613-8 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:106: finishing copy_from to smithi11731613-8 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 8 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:103: done (5 left) 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:104: done (4 left) 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:106: done (3 left) 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:109: setattr oid 24 current snap is 6 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:107: finishing write tid 1 to smithi11731613-39 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:110: read oid 35 snap 4 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:110: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 27 current snap is 6 2024-09-16T06:48:30.142 INFO:tasks.rados.rados.0.smithi117.stdout:111: seq_num 71 ranges {442368=425984,868352=573440,1441792=196608} 2024-09-16T06:48:30.151 INFO:tasks.rados.rados.0.smithi117.stdout:111: writing smithi11731613-27 from 442368 to 868352 tid 1 2024-09-16T06:48:30.157 INFO:tasks.rados.rados.0.smithi117.stdout:111: writing smithi11731613-27 from 868352 to 1441792 tid 2 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:111: writing smithi11731613-27 from 1441792 to 1638400 tid 3 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:112: read oid 33 snap -1 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:112: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:113: copy_from oid 17 from oid 13 current snap is 6 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:107: finishing write tid 2 to smithi11731613-39 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:107: finishing write tid 3 to smithi11731613-39 2024-09-16T06:48:30.160 INFO:tasks.rados.rados.0.smithi117.stdout:107: finishing write tid 4 to smithi11731613-39 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:107: oid 39 updating version 0 to 13 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:107: oid 39 updating version 13 to 14 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:107: oid 39 version 14 is already newer than 12 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 39 v 14 (ObjNum 70 snap 6 seq_num 70) dirty exists 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:107: left oid 39 (ObjNum 70 snap 6 seq_num 70) 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 24 v 16 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-16T06:48:30.161 INFO:tasks.rados.rados.0.smithi117.stdout:110: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-16T06:48:30.206 INFO:tasks.rados.rados.0.smithi117.stdout:111: finishing write tid 1 to smithi11731613-27 2024-09-16T06:48:30.207 INFO:tasks.rados.rados.0.smithi117.stdout:111: finishing write tid 2 to smithi11731613-27 2024-09-16T06:48:30.207 INFO:tasks.rados.rados.0.smithi117.stdout:111: finishing write tid 3 to smithi11731613-27 2024-09-16T06:48:30.207 INFO:tasks.rados.rados.0.smithi117.stdout:111: finishing write tid 4 to smithi11731613-27 2024-09-16T06:48:30.207 INFO:tasks.rados.rados.0.smithi117.stdout:112: expect (ObjNum 32 snap 0 seq_num 32) 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:111: finishing write tid 5 to smithi11731613-27 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:111: oid 27 updating version 0 to 20 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:111: oid 27 version 20 is already newer than 18 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:111: oid 27 updating version 20 to 21 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:111: oid 27 version 21 is already newer than 19 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 27 v 21 (ObjNum 71 snap 6 seq_num 71) dirty exists 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:111: left oid 27 (ObjNum 71 snap 6 seq_num 71) 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:113: finishing copy_from racing read to smithi11731613-17 2024-09-16T06:48:30.234 INFO:tasks.rados.rados.0.smithi117.stdout:113: finishing copy_from to smithi11731613-17 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 17 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:107: done (7 left) 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:109: done (6 left) 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:110: done (5 left) 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:111: done (4 left) 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:112: done (3 left) 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:113: done (2 left) 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 15 current snap is 6 2024-09-16T06:48:30.235 INFO:tasks.rados.rados.0.smithi117.stdout:114: seq_num 72 ranges {245760=442368,688128=720896,1409024=49152} 2024-09-16T06:48:30.236 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:30.235+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:30.243 INFO:tasks.rados.rados.0.smithi117.stdout:114: writing smithi11731613-15 from 245760 to 688128 tid 1 2024-09-16T06:48:30.253 INFO:tasks.rados.rados.0.smithi117.stdout:114: writing smithi11731613-15 from 688128 to 1409024 tid 2 2024-09-16T06:48:30.254 INFO:tasks.rados.rados.0.smithi117.stdout:114: writing smithi11731613-15 from 1409024 to 1458176 tid 3 2024-09-16T06:48:30.254 INFO:tasks.rados.rados.0.smithi117.stdout:115: read oid 7 snap -1 2024-09-16T06:48:30.254 INFO:tasks.rados.rados.0.smithi117.stdout:115: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:48:30.254 INFO:tasks.rados.rados.0.smithi117.stdout:116: read oid 8 snap -1 2024-09-16T06:48:30.254 INFO:tasks.rados.rados.0.smithi117.stdout:116: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:30.254 INFO:tasks.rados.rados.0.smithi117.stdout:117: snap_create 2024-09-16T06:48:30.255 INFO:tasks.rados.rados.0.smithi117.stdout:114: finishing write tid 1 to smithi11731613-15 2024-09-16T06:48:30.259 INFO:tasks.rados.rados.0.smithi117.stdout:116: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:30.271 INFO:tasks.rados.rados.0.smithi117.stdout:115: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: finishing write tid 2 to smithi11731613-15 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: finishing write tid 3 to smithi11731613-15 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: finishing write tid 4 to smithi11731613-15 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: finishing write tid 5 to smithi11731613-15 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: oid 15 updating version 0 to 25 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: oid 15 version 25 is already newer than 24 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: oid 15 version 25 is already newer than 23 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:114: oid 15 updating version 25 to 26 2024-09-16T06:48:30.273 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 15 v 26 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-16T06:48:30.274 INFO:tasks.rados.rados.0.smithi117.stdout:114: left oid 15 (ObjNum 72 snap 6 seq_num 72) 2024-09-16T06:48:30.313 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:30.313 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:30.327 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:48:30.338 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:30.336+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:30.344 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:30.344 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:30.348 INFO:tasks.rados.rados.0.smithi117.stdout:108: finishing copy_from racing read to smithi11731613-30 2024-09-16T06:48:30.350 INFO:tasks.rados.rados.0.smithi117.stdout:108: finishing copy_from to smithi11731613-30 2024-09-16T06:48:30.350 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:30.369 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:30.369 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:30.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:30.392 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:30.416 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:30.439 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:30.436+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:30.440 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:30.440 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:30.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:30.468 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:30.492 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:30.493 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:30.495 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:30.495 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:30.506 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:48:30.525 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:30.525 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:30.540 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:30.540+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:30.553 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:30.553 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:30.585 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:30.585 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:30.615 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:30.631 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:4 3.2s0 3:42d6f0b3:::smithi11731613-2:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892106+0000", 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7386050820000003, 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013862718, 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892106+0000", 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892106+0000", 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892103+0000", 2024-09-16T06:48:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892265+0000", 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000161636 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892292+0000", 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6716e-05 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892321+0000", 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9516e-05 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892606+0000", 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028500200000000002 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892925+0000", 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031936399999999999 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.894225+0000", 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012999140000000001 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896826+0000", 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026006570000000001 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905794+0000", 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089676909999999999 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905968+0000", 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000174649 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:5 3.2s0 3:42d6f0b3:::smithi11731613-2:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7383354310000003, 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029483957000000002, 2024-09-16T06:48:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:48:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892373+0000", 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892385+0000", 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1939999999999999e-05 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892403+0000", 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8202999999999998e-05 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892421+0000", 2024-09-16T06:48:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8189000000000002e-05 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895208+0000", 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786527 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895392+0000", 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000184255 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896349+0000", 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00095644999999999997 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902998+0000", 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066495039999999997 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921651+0000", 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018652716 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921859+0000", 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208359 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:6 3.2s0 3:42d6f0b3:::smithi11731613-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:30.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7382542519999999, 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034217268000000002, 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892455+0000", 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892464+0000", 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.67e-06 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892484+0000", 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0265999999999999e-05 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892500+0000", 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.596e-05 2024-09-16T06:48:30.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897287+0000", 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047868049999999999 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897527+0000", 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023998800000000001 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922181+0000", 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024653615 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922389+0000", 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020808600000000001 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926674+0000", 2024-09-16T06:48:30.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042850639999999999 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:43 3.2s0 3:4636348c:::smithi11731613-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7250361989999998, 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020346564000000001, 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905671+0000", 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909244+0000", 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035731700000000001 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909296+0000", 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2352999999999998e-05 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909321+0000", 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4813000000000001e-05 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909751+0000", 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042994099999999998 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.910155+0000", 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040433999999999999 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912510+0000", 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002354929 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924742+0000", 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012231967 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.925839+0000", 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001096302 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926021+0000", 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000182492 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:44 3.2s0 3:4636348c:::smithi11731613-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7213505690000002, 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034540078000000002, 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909357+0000", 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909370+0000", 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2583e-05 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909390+0000", 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9930000000000001e-05 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909421+0000", 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1285999999999998e-05 2024-09-16T06:48:30.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913550+0000", 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041288399999999999 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913768+0000", 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000217691 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914791+0000", 2024-09-16T06:48:30.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010230770000000001 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924746+0000", 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099553599999999999 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943736+0000", 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018990000999999999 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943900+0000", 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016405 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:45 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7212756330000003, 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040625472000000003, 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909434+0000", 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909441+0000", 2024-09-16T06:48:30.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0330000000000003e-06 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909460+0000", 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8674000000000001e-05 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909473+0000", 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3485999999999999e-05 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919590+0000", 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010117482000000001 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919812+0000", 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000221389 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944680+0000", 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024868369000000001 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944880+0000", 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019949900000000001 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.950061+0000", 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051807770000000001 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 35/32 ECSubWrite(tid=4, reqid=client.4511.0:40, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:30.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.915548+0000", 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7151625920000004, 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012162571, 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915548+0000", 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915548+0000", 2024-09-16T06:48:30.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915546+0000", 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915901+0000", 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035479100000000003 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916173+0000", 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027175799999999998 2024-09-16T06:48:30.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916190+0000", 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7054e-05 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916302+0000", 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112642 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916324+0000", 2024-09-16T06:48:30.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1673000000000001e-05 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927667+0000", 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011342703000000001 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927711+0000", 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3930000000000001e-05 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 35/32 ECSubWrite(tid=5, reqid=client.4511.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.915913+0000", 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7147972439999997, 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011672927, 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915913+0000", 2024-09-16T06:48:30.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915913+0000", 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915912+0000", 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916061+0000", 2024-09-16T06:48:30.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014851600000000001 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916234+0000", 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000173468 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916249+0000", 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4884e-05 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916436+0000", 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018702200000000001 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916459+0000", 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2413e-05 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927521+0000", 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011061807999999999 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927586+0000", 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5833000000000005e-05 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:72 3.2s0 3:4eee07bc:::smithi11731613-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.956955+0000", 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.6737561119999995, 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010281281, 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 72 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956955+0000", 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956955+0000", 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956954+0000", 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956960+0000", 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9680000000000003e-06 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956976+0000", 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6832000000000001e-05 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956989+0000", 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3114e-05 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.961169+0000", 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041798920000000002 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.961346+0000", 2024-09-16T06:48:30.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017679699999999999 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.965220+0000", 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038740300000000001 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.965451+0000", 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000230991 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.967236+0000", 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017846819999999999 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:81 3.2s0 3:476f74d7:::smithi11731613-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.969996+0000", 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.6607145899999995, 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011260137, 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 81 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.969996+0000", 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.969996+0000", 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.969995+0000", 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.970000+0000", 2024-09-16T06:48:30.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.69e-06 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.970008+0000", 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.452e-06 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.970014+0000", 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.088e-06 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.973521+0000", 2024-09-16T06:48:30.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003507212 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.973627+0000", 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010581099999999999 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.979248+0000", 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056215450000000004 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.979423+0000", 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017510200000000001 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.981256+0000", 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001832867 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:116 3.2s0 3:47b7cdf1:::smithi11731613-39:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.018607+0000", 2024-09-16T06:48:30.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.6121040200000003, 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010389344999999999, 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018607+0000", 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018607+0000", 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018605+0000", 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018613+0000", 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7039999999999994e-06 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018641+0000", 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7966e-05 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018652+0000", 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0732e-05 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021774+0000", 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031222390000000002 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021937+0000", 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016321900000000001 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.022532+0000", 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059462800000000004 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027450+0000", 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049179619999999997 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.028898+0000", 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001447771 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.028996+0000", 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8309999999999999e-05 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:117 3.2s0 3:47b7cdf1:::smithi11731613-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.018679+0000", 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.612031258, 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012405204, 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018679+0000", 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018679+0000", 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018678+0000", 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018684+0000", 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8039999999999998e-06 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018700+0000", 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6130999999999999e-05 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018713+0000", 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.238e-05 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.022854+0000", 2024-09-16T06:48:30.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041413459999999997 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.022975+0000", 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012055199999999999 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.029496+0000", 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006520896 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.029605+0000", 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109688 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031085+0000", 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001479403 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:230 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.6751145320000003, 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0201987, 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 230 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955593+0000", 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955613+0000", 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0630000000000001e-05 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955662+0000", 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8702999999999998e-05 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955689+0000", 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.73e-05 2024-09-16T06:48:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955849+0000", 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015934200000000001 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955874+0000", 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.548e-05 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966673+0000", 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010799329999999999 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966857+0000", 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018410900000000001 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975795+0000", 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089374099999999998 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:231 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.674988023, 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021272859000000002, 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 231 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955721+0000", 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955733+0000", 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2129e-05 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955755+0000", 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1423000000000001e-05 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955770+0000", 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4790000000000001e-05 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955974+0000", 2024-09-16T06:48:30.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020403199999999999 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955982+0000", 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6940000000000001e-06 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.967793+0000", 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011811056 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.968023+0000", 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023004500000000001 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976996+0000", 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089721929999999998 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:232 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.674763478, 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022137836000000001, 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 232 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955946+0000", 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955958+0000", 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-05 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955977+0000", 2024-09-16T06:48:30.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.912e-05 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955993+0000", 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5894000000000001e-05 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956101+0000", 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107402 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956123+0000", 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2056999999999999e-05 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971700+0000", 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015577155000000001 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971931+0000", 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023091400000000001 2024-09-16T06:48:30.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.978085+0000", 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061543179999999998 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:298 3.2s0 3:47b7cdf1:::smithi11731613-39:head [append 0~802816] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.127256+0000", 2024-09-16T06:48:30.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.50345419700000005, 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012207569999999999, 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 298 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.127256+0000", 2024-09-16T06:48:30.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.127256+0000", 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.127254+0000", 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128653+0000", 2024-09-16T06:48:30.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013990299999999999 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128685+0000", 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1721999999999999e-05 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128701+0000", 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5787e-05 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128802+0000", 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100853 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128962+0000", 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016078199999999999 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.130997+0000", 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020347849999999999 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134383+0000", 2024-09-16T06:48:30.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003385435 2024-09-16T06:48:30.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.139275+0000", 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048926799999999999 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.139464+0000", 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000188822 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:301 3.2s0 3:47b7cdf1:::smithi11731613-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.134619+0000", 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49609193699999998, 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011711747, 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 301 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134619+0000", 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134619+0000", 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134617+0000", 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:30.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134625+0000", 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6699999999999994e-06 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134644+0000", 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134663+0000", 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9059000000000002e-05 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.140923+0000", 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062592910000000002 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.141178+0000", 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025534400000000001 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.144207+0000", 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030284499999999998 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.144334+0000", 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000127179 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.146331+0000", 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019967909999999999 2024-09-16T06:48:30.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:312 3.2s0 3:476f74d7:::smithi11731613-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.161498+0000", 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.46921222000000001, 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010992791999999999, 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 312 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.161498+0000", 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.161498+0000", 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.161497+0000", 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.161506+0000", 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6400000000000003e-06 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.161524+0000", 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893e-05 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.161540+0000", 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6322999999999999e-05 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.165729+0000", 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041884510000000002 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.165832+0000", 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102963 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.169908+0000", 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040762410000000004 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.170163+0000", 2024-09-16T06:48:30.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025474500000000001 2024-09-16T06:48:30.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.172491+0000", 2024-09-16T06:48:30.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023287020000000002 2024-09-16T06:48:30.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.732 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:321 3.2s0 3:4636348c:::smithi11731613-15:head [setxattr _header (56) in=7b] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:30.732 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.255552+0000", 2024-09-16T06:48:30.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.375158993, 2024-09-16T06:48:30.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010411416, 2024-09-16T06:48:30.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:30.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 321 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255552+0000", 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255552+0000", 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255551+0000", 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255556+0000", 2024-09-16T06:48:30.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.831e-06 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255564+0000", 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.473e-06 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255571+0000", 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2570000000000004e-06 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.259566+0000", 2024-09-16T06:48:30.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039953480000000001 2024-09-16T06:48:30.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.259695+0000", 2024-09-16T06:48:30.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000128823 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.260286+0000", 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059164899999999997 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:30.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.261045+0000", 2024-09-16T06:48:30.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075893 2024-09-16T06:48:30.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:30.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.265856+0000", 2024-09-16T06:48:30.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048107339999999997 2024-09-16T06:48:30.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.265963+0000", 2024-09-16T06:48:30.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010704099999999999 2024-09-16T06:48:30.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:322 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:30.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.255575+0000", 2024-09-16T06:48:30.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.37513593299999998, 2024-09-16T06:48:30.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012092308, 2024-09-16T06:48:30.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:30.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:30.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:30.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:30.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:30.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 322 2024-09-16T06:48:30.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:30.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:30.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255575+0000", 2024-09-16T06:48:30.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:30.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255575+0000", 2024-09-16T06:48:30.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:30.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:30.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255574+0000", 2024-09-16T06:48:30.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:30.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:30.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255577+0000", 2024-09-16T06:48:30.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-09-16T06:48:30.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255583+0000", 2024-09-16T06:48:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.686e-06 2024-09-16T06:48:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255589+0000", 2024-09-16T06:48:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7400000000000001e-06 2024-09-16T06:48:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.260880+0000", 2024-09-16T06:48:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052912239999999998 2024-09-16T06:48:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.260988+0000", 2024-09-16T06:48:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000108108 2024-09-16T06:48:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.266365+0000", 2024-09-16T06:48:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053770550000000004 2024-09-16T06:48:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:30.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.266465+0000", 2024-09-16T06:48:30.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9570999999999999e-05 2024-09-16T06:48:30.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:30.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:30.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:30.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.267667+0000", 2024-09-16T06:48:30.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001202079 2024-09-16T06:48:30.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:30.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:30.756 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:30.758 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:30.759 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:30.760 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:30.760 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:30.761 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:48:30.762 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:30.763 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:30.763 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:30.764 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:30.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:30.728+0000 7fbd22a8c640 0 -- 172.21.15.100:0/901951846 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7fbd1c221e60 msgr2=0x7fbd1c224270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:30.765 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:30.756+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:30.859 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:30.856+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:30.896 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:30.896 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:30.912 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:48:30.960 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:30.960+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:30.974 INFO:tasks.rados.rados.0.smithi117.stdout:108: done (5 left) 2024-09-16T06:48:30.974 INFO:tasks.rados.rados.0.smithi117.stdout:114: done (4 left) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:115: done (3 left) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:116: done (2 left) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:117: done (1 left) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:118: read oid 31 snap -1 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:118: expect (ObjNum 61 snap 4 seq_num 61) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:119: rmattr oid 43 current snap is 7 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:119: done (2 left) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:120: read oid 10 snap -1 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:120: expect (ObjNum 54 snap 2 seq_num 54) 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:121: copy_from oid 33 from oid 42 current snap is 7 2024-09-16T06:48:30.975 INFO:tasks.rados.rados.0.smithi117.stdout:122: snap_remove snap 0 2024-09-16T06:48:31.037 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.038 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.052 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:48:31.097 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:31.096+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:31.161 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:31.160+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:31.175 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.175 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.189 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:48:31.263 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:31.260+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:31.358 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.359 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.364 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:31.364+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:31.372 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:48:31.465 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:31.464+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:31.490 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.490 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.504 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:48:31.566 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:31.568+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:31.667 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:31.667+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:31.668 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.668 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.682 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:48:31.768 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:31.763+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:31.804 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.804 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.807 INFO:tasks.rados.rados.0.smithi117.stdout:122: done (4 left) 2024-09-16T06:48:31.807 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 41 current snap is 7 2024-09-16T06:48:31.807 INFO:tasks.rados.rados.0.smithi117.stdout:123: seq_num 73 ranges {2785280=458752,3244032=475136,3719168=294912} 2024-09-16T06:48:31.825 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:48:31.846 INFO:tasks.rados.rados.0.smithi117.stdout:123: writing smithi11731613-41 from 2785280 to 3244032 tid 1 2024-09-16T06:48:31.851 INFO:tasks.rados.rados.0.smithi117.stdout:123: writing smithi11731613-41 from 3244032 to 3719168 tid 2 2024-09-16T06:48:31.853 INFO:tasks.rados.rados.0.smithi117.stdout:123: writing smithi11731613-41 from 3719168 to 4014080 tid 3 2024-09-16T06:48:31.854 INFO:tasks.rados.rados.0.smithi117.stdout:120: expect (ObjNum 54 snap 2 seq_num 54) 2024-09-16T06:48:31.868 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:31.868+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:31.872 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:31.872 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:31.891 INFO:tasks.rados.rados.0.smithi117.stdout:121: finishing copy_from racing read to smithi11731613-33 2024-09-16T06:48:31.891 INFO:tasks.rados.rados.0.smithi117.stdout:118: expect (ObjNum 61 snap 4 seq_num 61) 2024-09-16T06:48:31.942 INFO:tasks.rados.rados.0.smithi117.stdout:121: finishing copy_from to smithi11731613-33 2024-09-16T06:48:31.943 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 33 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-16T06:48:31.943 INFO:tasks.rados.rados.0.smithi117.stdout:105: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-16T06:48:31.953 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:31.953 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:31.963 INFO:tasks.rados.rados.0.smithi117.stdout:123: finishing write tid 1 to smithi11731613-41 2024-09-16T06:48:31.963 INFO:tasks.rados.rados.0.smithi117.stdout:123: finishing write tid 2 to smithi11731613-41 2024-09-16T06:48:31.963 INFO:tasks.rados.rados.0.smithi117.stdout:105: done (4 left) 2024-09-16T06:48:31.963 INFO:tasks.rados.rados.0.smithi117.stdout:118: done (3 left) 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:120: done (2 left) 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:121: done (1 left) 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:124: snap_create 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: finishing write tid 3 to smithi11731613-41 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: finishing write tid 4 to smithi11731613-41 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: finishing write tid 5 to smithi11731613-41 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: oid 41 updating version 0 to 40 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: oid 41 updating version 40 to 41 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: oid 41 updating version 41 to 42 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: oid 41 version 42 is already newer than 39 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 41 v 42 (ObjNum 73 snap 7 seq_num 73) dirty exists 2024-09-16T06:48:31.964 INFO:tasks.rados.rados.0.smithi117.stdout:123: left oid 41 (ObjNum 73 snap 7 seq_num 73) 2024-09-16T06:48:31.969 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:48:31.969 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:31.968+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:31.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:48:32.069 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:32.067+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:32.137 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:32.137 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:32.151 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:48:32.172 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:32.168+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:32.271 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:32.268+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:32.275 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:32.276 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:32.290 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:48:32.372 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:32.367+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:32.460 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:32.460 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:32.473 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:32.471+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:32.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:48:32.574 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:32.572+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:32.596 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:32.596 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:32.611 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:48:32.675 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:32.671+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:32.739 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:32.739 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:32.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:48:32.776 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:32.772+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:32.804 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:48:32.810 INFO:tasks.rados.rados.0.smithi117.stdout:123: done (1 left) 2024-09-16T06:48:32.810 INFO:tasks.rados.rados.0.smithi117.stdout:124: done (0 left) 2024-09-16T06:48:32.810 INFO:tasks.rados.rados.0.smithi117.stdout:125: snap_create 2024-09-16T06:48:32.877 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:32.876+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:32.877 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:32.877 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:32.892 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:48:32.978 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:32.976+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:33.058 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.058 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:48:33.078 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:33.079+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:33.179 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:33.175+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:33.195 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.195 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.209 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:48:33.280 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:33.280+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:33.329 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.329 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:48:33.388 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:33.388+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:33.459 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.459 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:48:33.481 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:33.479+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:33.582 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:33.579+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:33.647 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.647 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:48:33.683 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:33.683+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:33.784 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:33.783+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:33.784 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.785 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.799 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:48:33.812 INFO:tasks.rados.rados.0.smithi117.stdout:125: done (0 left) 2024-09-16T06:48:33.812 INFO:tasks.rados.rados.0.smithi117.stdout:126: snap_create 2024-09-16T06:48:33.885 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:33.883+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:33.905 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:33.922 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:33.922 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:33.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:48:33.985 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:33.983+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:34.065 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:34.065 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:34.086 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:34.084+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:34.089 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:34.090 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:34.099 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:34.099 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:34.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:48:34.115 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:34.115 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:34.140 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:34.140 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:34.161 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:34.161 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:34.187 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:34.187 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:34.187 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:34.184+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:34.230 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:34.230 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:34.244 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:48:34.288 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:34.287+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:34.314 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:34.314 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:34.349 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:34.384 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:34.385 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:34.389 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:34.387+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:34.403 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:34.404 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:34.412 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:34.412 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:34.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:48:34.440 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:34.441 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:34.473 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:34.473 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:34.490 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:34.487+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:34.502 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:34.502 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:34.535 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:34.535 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:34.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:34.540+0000 7ff31bfff640 0 -- 172.21.15.100:0/3291692813 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7ff31c21d5d0 msgr2=0x7ff31c21f9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:34.551 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:34.551 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:34.551 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:34.551 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:34.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:48:34.575 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:34.576 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:34.591 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:34.588+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:34.608 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:34.682 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:34.683 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:34.692 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:34.692+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:34.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:48:34.793 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:34.791+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:34.815 INFO:tasks.rados.rados.0.smithi117.stdout:126: done (0 left) 2024-09-16T06:48:34.815 INFO:tasks.rados.rados.0.smithi117.stdout:127: rmattr oid 20 current snap is 10 2024-09-16T06:48:34.815 INFO:tasks.rados.rados.0.smithi117.stdout:127: done (0 left) 2024-09-16T06:48:34.815 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 33 current snap is 10 2024-09-16T06:48:34.815 INFO:tasks.rados.rados.0.smithi117.stdout:128: seq_num 74 ranges {278528=475136,753664=622592,1376256=147456} 2024-09-16T06:48:34.821 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:10 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:34.824 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.925730561, 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025558260999999999, 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896131+0000", 2024-09-16T06:48:34.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896384+0000", 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025249400000000003 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896423+0000", 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9551999999999997e-05 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896446+0000", 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2657999999999999e-05 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896601+0000", 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015511400000000001 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896938+0000", 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000336779 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899233+0000", 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022945840000000001 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903156+0000", 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039230970000000004 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921526+0000", 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018370496 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921693+0000", 2024-09-16T06:48:34.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000166916 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:11 3.4s0 3:2be2ffb2:::smithi11731613-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.925377957, 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025954171000000002, 2024-09-16T06:48:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896485+0000", 2024-09-16T06:48:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896497+0000", 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1669e-05 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896518+0000", 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1180999999999999e-05 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896534+0000", 2024-09-16T06:48:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5732000000000001e-05 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900249+0000", 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037148250000000002 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900475+0000", 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022637300000000001 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901657+0000", 2024-09-16T06:48:34.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001182249 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.909750+0000", 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0080923149999999992 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922220+0000", 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012470623 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922442+0000", 2024-09-16T06:48:34.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022146399999999999 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:12 3.4s0 3:2be2ffb2:::smithi11731613-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.925276597, 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032662215000000001, 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 12 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896587+0000", 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896595+0000", 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9389999999999997e-06 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896616+0000", 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0571e-05 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896631+0000", 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5315000000000002e-05 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057790200000000002 2024-09-16T06:48:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902594+0000", 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018354499999999999 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922598+0000", 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020004152000000001 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923951+0000", 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001352771 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929251+0000", 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005300389 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:34 3.4s0 3:2123e90c:::smithi11731613-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:34.842 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:34.843 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.920774316999999, 2024-09-16T06:48:34.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031689311999999997, 2024-09-16T06:48:34.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 34 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:34.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901089+0000", 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901956+0000", 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086772399999999995 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901994+0000", 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7880000000000003e-05 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902023+0000", 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8479999999999998e-05 2024-09-16T06:48:34.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902695+0000", 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067224400000000001 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902713+0000", 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7719000000000001e-05 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.910786+0000", 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00807276 2024-09-16T06:48:34.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911114+0000", 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032852700000000001 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914294+0000", 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003180113 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918604+0000", 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043102669999999996 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932586+0000", 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013981214 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932780+0000", 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019474 2024-09-16T06:48:34.849 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:35 3.4s0 3:2123e90c:::smithi11731613-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.919819598, 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031230615999999999, 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.850 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 35 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902044+0000", 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902055+0000", 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0961999999999999e-05 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902075+0000", 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9748000000000001e-05 2024-09-16T06:48:34.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902088+0000", 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3791999999999999e-05 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001041583 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903146+0000", 2024-09-16T06:48:34.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6511000000000002e-05 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.915912+0000", 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012765472999999999 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916161+0000", 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024903700000000002 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917305+0000", 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001143813 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927287+0000", 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099818019999999997 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933082+0000", 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057955740000000004 2024-09-16T06:48:34.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933276+0000", 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019429399999999999 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.fs6 34/32 ECSubWrite(tid=2, reqid=client.4511.0:19, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902322+0000", 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.9195435, 2024-09-16T06:48:34.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010231046000000001, 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902322+0000", 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902322+0000", 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902320+0000", 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902440+0000", 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119268 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902575+0000", 2024-09-16T06:48:34.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013575300000000001 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902597+0000", 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1398000000000001e-05 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.904604+0000", 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020070029999999998 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.904626+0000", 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1824e-05 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912487+0000", 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078612830000000002 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912553+0000", 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5950000000000004e-05 2024-09-16T06:48:34.860 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:36 3.4s0 3:2123e90c:::smithi11731613-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.919397538, 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038105068999999998, 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.861 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 36 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902465+0000", 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902479+0000", 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3405e-05 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902529+0000", 2024-09-16T06:48:34.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9638999999999997e-05 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902555+0000", 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903331+0000", 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077605999999999997 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:34.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903346+0000", 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5362999999999998e-05 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918677+0000", 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015331205000000001 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919160+0000", 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00048271000000000002 2024-09-16T06:48:34.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936255+0000", 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017094597999999999 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936424+0000", 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168789 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940573+0000", 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041493600000000004 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:93 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.990651+0000", 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.831214182, 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011000630000000001, 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:34.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 93 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990651+0000", 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990651+0000", 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990650+0000", 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990657+0000", 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2740000000000002e-06 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990676+0000", 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8624e-05 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990689+0000", 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3141e-05 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.996905+0000", 2024-09-16T06:48:34.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062163640000000003 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.997144+0000", 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238765 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.998942+0000", 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001798062 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.999158+0000", 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021595100000000001 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.001652+0000", 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002493706 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.872 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:135 3.4s0 3:2cfe9c23:::smithi11731613-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.050267+0000", 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.771598712999999, 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010744566000000001, 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 135 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050267+0000", 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050267+0000", 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050266+0000", 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050272+0000", 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6989999999999997e-06 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050291+0000", 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8847000000000001e-05 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050310+0000", 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8442999999999999e-05 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054567+0000", 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042575620000000003 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054688+0000", 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120741 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.059268+0000", 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045798139999999998 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.059338+0000", 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0190000000000004e-05 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061011+0000", 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001673384 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:227 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958230+0000", 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8636349939999999, 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0115925, 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 227 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958230+0000", 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958230+0000", 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958227+0000", 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958247+0000", 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0268999999999999e-05 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958285+0000", 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7698999999999999e-05 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958309+0000", 2024-09-16T06:48:34.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3538000000000001e-05 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958489+0000", 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017992799999999999 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958510+0000", 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1220999999999999e-05 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960761+0000", 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022513429999999998 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960816+0000", 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4796999999999997e-05 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961347+0000", 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00053147800000000005 2024-09-16T06:48:34.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964120+0000", 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027730979999999999 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.969759+0000", 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056390040000000004 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.969823+0000", 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3411999999999995e-05 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:228 3.4s0 3:2be2ffb2:::smithi11731613-4:head [rollback 5] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8635130220000002, 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015696132000000002, 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 228 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:34.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958355+0000", 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3280000000000002e-06 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958364+0000", 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6410000000000008e-06 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958370+0000", 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0009999999999998e-06 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958705+0000", 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033533 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958720+0000", 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4976e-05 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961715+0000", 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029947229999999999 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961796+0000", 2024-09-16T06:48:34.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1070999999999997e-05 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962213+0000", 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00041693299999999997 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964999+0000", 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027864159999999999 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974015+0000", 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090157589999999999 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974049+0000", 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3501999999999999e-05 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:229 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8633713150000002, 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015649126999999999, 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 229 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:34.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958497+0000", 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4709999999999999e-06 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958504+0000", 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1500000000000002e-06 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958510+0000", 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9399999999999999e-06 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958785+0000", 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027471700000000001 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958790+0000", 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9470000000000002e-06 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962470+0000", 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036807070000000001 2024-09-16T06:48:34.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962542+0000", 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1168999999999996e-05 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962912+0000", 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037075200000000002 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965035+0000", 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021227419999999999 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974112+0000", 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090767599999999997 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974143+0000", 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1315000000000001e-05 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:237 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8633464100000001, 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013107461000000001, 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 237 2024-09-16T06:48:34.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958521+0000", 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6469999999999998e-06 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958529+0000", 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8830000000000006e-06 2024-09-16T06:48:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958535+0000", 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4410000000000001e-06 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958587+0000", 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2215000000000001e-05 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:34.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958594+0000", 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8589999999999999e-06 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960575+0000", 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019818380000000001 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960703+0000", 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012750899999999999 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961363+0000", 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00066002300000000003 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964110+0000", 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027467659999999999 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.971588+0000", 2024-09-16T06:48:34.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074782549999999996 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.971626+0000", 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8476999999999998e-05 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:238 3.1s0 3:825f7e25:::smithi11731613-3:head [rollback 2] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.863321096, 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014029764, 2024-09-16T06:48:34.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 238 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958547+0000", 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2629999999999999e-06 2024-09-16T06:48:34.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958554+0000", 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0480000000000001e-06 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958561+0000", 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2919999999999998e-06 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958670+0000", 2024-09-16T06:48:34.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010880800000000001 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958682+0000", 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1966e-05 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961922+0000", 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032400459999999999 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961990+0000", 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7710000000000001e-05 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962713+0000", 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000722738 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965012+0000", 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022997120000000002 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972544+0000", 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007531055 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972574+0000", 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0550999999999999e-05 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:239 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8632990889999999, 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014095699999999999, 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 239 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958570+0000", 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7950000000000001e-06 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958576+0000", 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958583+0000", 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2629999999999997e-06 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958766+0000", 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018306299999999999 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958783+0000", 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6801000000000001e-05 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963004+0000", 2024-09-16T06:48:34.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004221168 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963078+0000", 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3843000000000004e-05 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963329+0000", 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025082499999999998 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965054+0000", 2024-09-16T06:48:34.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017253220000000001 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972634+0000", 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075798469999999998 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972662+0000", 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8232999999999999e-05 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.915 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:240 3.4s0 3:2ae27109:::smithi11731613-31:head [append 0~655360] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.975800+0000", 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.846065544, 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055275000000001, 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 240 2024-09-16T06:48:34.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.975800+0000", 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.975800+0000", 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.975799+0000", 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977071+0000", 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001272537 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977089+0000", 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7988000000000001e-05 2024-09-16T06:48:34.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977100+0000", 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0750999999999999e-05 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977132+0000", 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2286999999999998e-05 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977189+0000", 2024-09-16T06:48:34.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6771000000000002e-05 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977931+0000", 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00074231899999999996 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.980412+0000", 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002481249 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.987684+0000", 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0072718000000000001 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.987855+0000", 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017083800000000001 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:243 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8398874750000003, 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01277557, 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 243 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:34.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981977+0000", 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981982+0000", 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0669999999999999e-06 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981993+0000", 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0535e-05 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.982010+0000", 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7210999999999998e-05 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.984848+0000", 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028378570000000001 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.984935+0000", 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7505999999999999e-05 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989994+0000", 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050582639999999998 2024-09-16T06:48:34.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990209+0000", 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021580699999999999 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994753+0000", 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045440899999999998 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:285 3.4s0 3:29582896:::smithi11731613-28:6 [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.007628+0000", 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8142371590000002, 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011462415, 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 285 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007628+0000", 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007628+0000", 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007627+0000", 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:34.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007635+0000", 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6930000000000007e-06 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007653+0000", 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7788e-05 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007674+0000", 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0925999999999999e-05 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.011982+0000", 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043087990000000003 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012289+0000", 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030681099999999999 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.019091+0000", 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068014649999999996 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:288 3.4s0 3:2be2ffb2:::smithi11731613-4:head [rollback 6] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.103294+0000", 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.7185709710000001, 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01131003, 2024-09-16T06:48:34.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 288 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103294+0000", 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103294+0000", 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103293+0000", 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103302+0000", 2024-09-16T06:48:34.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1719999999999996e-06 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103321+0000", 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814999999999999e-05 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103334+0000", 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3003000000000001e-05 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.106066+0000", 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027321429999999998 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.106345+0000", 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027896300000000002 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108016+0000", 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016707989999999999 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.110649+0000", 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026328110000000001 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.114434+0000", 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037854910000000002 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.114604+0000", 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017000000000000001 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:289 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.7184577509999999, 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013340714, 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 289 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.937 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103406+0000", 2024-09-16T06:48:34.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103414+0000", 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7219999999999999e-06 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103437+0000", 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2785e-05 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:34.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103449+0000", 2024-09-16T06:48:34.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2629000000000001e-05 2024-09-16T06:48:34.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108874+0000", 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054250820000000003 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.109046+0000", 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000171173 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:34.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.109782+0000", 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073599699999999995 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.111135+0000", 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013535349999999999 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.116590+0000", 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054547659999999998 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:34.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.116748+0000", 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015846100000000001 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:34.943 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:34.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:48:34.946 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:48:34.947 INFO:tasks.rados.rados.0.smithi117.stdout:128: writing smithi11731613-33 from 278528 to 753664 tid 1 2024-09-16T06:48:34.947 INFO:tasks.rados.rados.0.smithi117.stdout:128: writing smithi11731613-33 from 753664 to 1376256 tid 2 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:128: writing smithi11731613-33 from 1376256 to 1523712 tid 3 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:128: finishing write tid 1 to smithi11731613-33 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:129: read oid 1 snap -1 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:129: expect (ObjNum 55 snap 2 seq_num 55) 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:130: rollback oid 47 current snap is 10 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 47 to 2 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:131: delete oid 43 current snap is 10 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:130: finishing rollback tid 0 to smithi11731613-47 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:130: finishing rollback tid 1 to smithi11731613-47 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:130: finishing rollback tid 2 to smithi11731613-47 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 47 v 28 (ObjNum 52 snap 0 seq_num 52) dirty exists 2024-09-16T06:48:34.948 INFO:tasks.rados.rados.0.smithi117.stdout:130: done (3 left) 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:131: done (2 left) 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:132: rmattr oid 26 current snap is 10 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:132: done (2 left) 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:133: rollback oid 10 current snap is 10 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 10 to 8 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:134: snap_create 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:129: expect (ObjNum 55 snap 2 seq_num 55) 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:128: finishing write tid 2 to smithi11731613-33 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:128: finishing write tid 3 to smithi11731613-33 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:128: finishing write tid 4 to smithi11731613-33 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:128: finishing write tid 5 to smithi11731613-33 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:128: oid 33 updating version 0 to 34 2024-09-16T06:48:34.949 INFO:tasks.rados.rados.0.smithi117.stdout:128: oid 33 version 34 is already newer than 31 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:128: oid 33 version 34 is already newer than 32 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:128: oid 33 version 34 is already newer than 33 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 33 v 34 (ObjNum 74 snap 10 seq_num 74) dirty exists 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:128: left oid 33 (ObjNum 74 snap 10 seq_num 74) 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:133: finishing rollback tid 0 to smithi11731613-10 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:133: finishing rollback tid 1 to smithi11731613-10 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:133: finishing rollback tid 2 to smithi11731613-10 2024-09-16T06:48:34.950 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 47 (ObjNum 54 snap 2 seq_num 54) dirty exists 2024-09-16T06:48:34.950 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:34.944+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:35.047 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:35.044+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:35.117 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:35.118 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:35.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:48:35.138 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:35.138 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:35.139 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T06:48:35.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T06:48:35.147 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:35.144+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:35.249 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:35.247+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:35.253 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:35.253 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:35.253 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:35.253 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:35.253 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:35.267 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:48:35.349 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:35.348+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:35.388 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 34/32 ECSubWrite(tid=0, reqid=client.4511.0:1, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.895950+0000", 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.492543216, 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039238140000000003, 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895950+0000", 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895950+0000", 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895947+0000", 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896088+0000", 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014060799999999999 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896201+0000", 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112959 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896234+0000", 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3322999999999999e-05 2024-09-16T06:48:35.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896371+0000", 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013665300000000001 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896394+0000", 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3331000000000001e-05 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899802+0000", 2024-09-16T06:48:35.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034075770000000002 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899874+0000", 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2285000000000003e-05 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 35/32 ECSubWrite(tid=4, reqid=client.4511.0:49, at_version=35'3, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.927479+0000", 2024-09-16T06:48:35.393 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.461014433999999, 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036545549999999999, 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927479+0000", 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927479+0000", 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927477+0000", 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927759+0000", 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028246800000000002 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927860+0000", 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010074 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927885+0000", 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4947e-05 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928035+0000", 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014955200000000001 2024-09-16T06:48:35.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928058+0000", 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3000999999999999e-05 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.931078+0000", 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030205240000000001 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.931134+0000", 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5482999999999997e-05 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 35/32 ECSubWrite(tid=5, reqid=client.4511.0:50, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.928468+0000", 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.460025429, 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004872103, 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.398 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928468+0000", 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928468+0000", 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928466+0000", 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928521+0000", 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4593999999999999e-05 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928615+0000", 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4092999999999995e-05 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928723+0000", 2024-09-16T06:48:35.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010840600000000001 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929279+0000", 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055508 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929301+0000", 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2150999999999998e-05 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933291+0000", 2024-09-16T06:48:35.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039900680000000003 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933340+0000", 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9419999999999998e-05 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=1, reqid=client.4511.0:53, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.445889791999999, 2024-09-16T06:48:35.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.67331382100000003, 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:35.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942602+0000", 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942641+0000", 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9292999999999998e-05 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614766+0000", 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.67212478200000003 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614782+0000", 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5654999999999998e-05 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614883+0000", 2024-09-16T06:48:35.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010158 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614901+0000", 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7337999999999999e-05 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.615860+0000", 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00095891399999999995 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.615918+0000", 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7978e-05 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=157, reqid=client.4511.0:76, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.407 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.977358+0000", 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.411135254, 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048110519999999997, 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977358+0000", 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977358+0000", 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977356+0000", 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977962+0000", 2024-09-16T06:48:35.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00060570999999999997 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978057+0000", 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4975999999999993e-05 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978091+0000", 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3804999999999999e-05 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978191+0000", 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100482 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978206+0000", 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4752999999999999e-05 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982118+0000", 2024-09-16T06:48:35.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039119350000000001 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982169+0000", 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1134999999999997e-05 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=158, reqid=client.4511.0:77, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.978821+0000", 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.409672838000001, 2024-09-16T06:48:35.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034290509999999998, 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978821+0000", 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978821+0000", 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978819+0000", 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978877+0000", 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8062000000000001e-05 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979044+0000", 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016653300000000001 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979061+0000", 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6699999999999999e-05 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979168+0000", 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107475 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.979179+0000", 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1323e-05 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982191+0000", 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030112400000000001 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982250+0000", 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9039999999999997e-05 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=159, reqid=client.4511.0:118, at_version=35'3, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.027408+0000", 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.361085495999999, 2024-09-16T06:48:35.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042951179999999997, 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.027408+0000", 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.027408+0000", 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.027405+0000", 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028149+0000", 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000743912 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028253+0000", 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010416099999999999 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028280+0000", 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6308999999999998e-05 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028447+0000", 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000167658 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028474+0000", 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6792e-05 2024-09-16T06:48:35.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.031637+0000", 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031629269999999998 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.031703+0000", 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6196000000000001e-05 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=160, reqid=client.4511.0:119, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.359518367, 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050875180000000001, 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028974+0000", 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029016+0000", 2024-09-16T06:48:35.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2521000000000002e-05 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029189+0000", 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017280600000000001 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029213+0000", 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4394999999999999e-05 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029795+0000", 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00058197499999999998 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029816+0000", 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0650999999999999e-05 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.034007+0000", 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041915119999999997 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.034063+0000", 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5145999999999997e-05 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=18, reqid=client.4511.0:130, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.342870394, 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051140320000000001, 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045622+0000", 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:35.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046511+0000", 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00088898600000000003 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046635+0000", 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124404 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046683+0000", 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.761e-05 2024-09-16T06:48:35.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046815+0000", 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013226000000000001 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046828+0000", 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2502e-05 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050686+0000", 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038587869999999998 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050737+0000", 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0785000000000003e-05 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.429 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=19, reqid=client.4511.0:131, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.046717+0000", 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.341776620999999, 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043397979999999997, 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046717+0000", 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046717+0000", 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046716+0000", 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046757+0000", 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.07e-05 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046822+0000", 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5283999999999994e-05 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046834+0000", 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2310000000000001e-05 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047497+0000", 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00066222799999999995 2024-09-16T06:48:35.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047512+0000", 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5651e-05 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051012+0000", 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034997539999999999 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051057+0000", 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4702e-05 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.436 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=20, reqid=client.4511.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.336835637, 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061083889999999997, 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051650+0000", 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999924 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051678+0000", 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8297999999999999e-05 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051750+0000", 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1749999999999996e-05 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051777+0000", 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7169999999999999e-05 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051852+0000", 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4768e-05 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051866+0000", 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3893000000000001e-05 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057731+0000", 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00586551 2024-09-16T06:48:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057766+0000", 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4795999999999997e-05 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=25, reqid=client.4511.0:140, at_version=35'6, trim_to=0'0, roll_forward_to=35'3))", 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.061567+0000", 2024-09-16T06:48:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.326926969000001, 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047379930000000002, 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061567+0000", 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061567+0000", 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061566+0000", 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061587+0000", 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656000000000001e-05 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061663+0000", 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5850000000000001e-05 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061671+0000", 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0800000000000006e-06 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.062587+0000", 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00091552599999999995 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.062608+0000", 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1171000000000001e-05 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.066282+0000", 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036744220000000001 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.066305+0000", 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2220000000000001e-05 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=31, reqid=client.4511.0:161, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.873285260999999, 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054402210000000003, 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.446 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515206+0000", 2024-09-16T06:48:35.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515247+0000", 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0228999999999997e-05 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515342+0000", 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5420000000000005e-05 2024-09-16T06:48:35.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515356+0000", 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3952e-05 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516560+0000", 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012035839999999999 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516583+0000", 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3064000000000001e-05 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520594+0000", 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040113630000000004 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520648+0000", 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4425000000000002e-05 2024-09-16T06:48:35.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:221 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.4248323020000004, 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0098559149999999998, 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:35.451 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 221 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963659+0000", 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963670+0000", 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0423e-05 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963685+0000", 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5596000000000001e-05 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963698+0000", 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2413e-05 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963842+0000", 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144444 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963862+0000", 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9874999999999999e-05 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.967541+0000", 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036789790000000002 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.967631+0000", 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9548000000000004e-05 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.973517+0000", 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00588654 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:222 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.4247754969999997, 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010055573, 2024-09-16T06:48:35.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 222 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963717+0000", 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963728+0000", 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0609e-05 2024-09-16T06:48:35.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963739+0000", 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0829000000000001e-05 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963762+0000", 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2894000000000001e-05 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024621499999999999 2024-09-16T06:48:35.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964013+0000", 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7840000000000003e-06 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968066+0000", 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040535229999999998 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:35.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968114+0000", 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7788999999999999e-05 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.973774+0000", 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056596190000000003 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.461 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:223 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.424689077, 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011053090999999999, 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 223 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963803+0000", 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963818+0000", 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4983999999999999e-05 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963830+0000", 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1472e-05 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963839+0000", 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2420000000000006e-06 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964057+0000", 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217455 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964066+0000", 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9570000000000008e-06 2024-09-16T06:48:35.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968321+0000", 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042553460000000001 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968427+0000", 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105887 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974857+0000", 2024-09-16T06:48:35.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064307560000000001 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 40/37 ECSubWrite(tid=21, reqid=client.4511.0:219, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.3981453339999996, 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057451949999999998, 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990346+0000", 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990377+0000", 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0463999999999998e-05 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990469+0000", 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1932e-05 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990492+0000", 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3127999999999999e-05 2024-09-16T06:48:35.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990628+0000", 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000136212 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990653+0000", 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5289000000000001e-05 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.996040+0000", 2024-09-16T06:48:35.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053864309999999997 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.996093+0000", 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3735e-05 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 40/37 ECSubWrite(tid=59, reqid=client.4511.0:233, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.39477136, 2024-09-16T06:48:35.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064058910000000004, 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993721+0000", 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994930+0000", 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001209803 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995211+0000", 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028033799999999998 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995239+0000", 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8453999999999999e-05 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995377+0000", 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000137797 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995407+0000", 2024-09-16T06:48:35.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0154999999999998e-05 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000083+0000", 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046756920000000004 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000128+0000", 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5241999999999998e-05 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:286 3.bs0 3:db3fccb6:::smithi11731613-6:head [stat,delete] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3810606029999999, 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019136058000000001, 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 286 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.476 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007430+0000", 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007447+0000", 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7146000000000002e-05 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007483+0000", 2024-09-16T06:48:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-05 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007508+0000", 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5701e-05 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007675+0000", 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016656 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007699+0000", 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3742000000000001e-05 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012471+0000", 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047720949999999996 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012760+0000", 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028969899999999998 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.014682+0000", 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019216789999999999 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.017646+0000", 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029635830000000001 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.026466+0000", 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0088207990000000007 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.026569+0000", 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010246099999999999 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 47/37 ECSubWrite(tid=92, reqid=client.4511.0:356, at_version=47'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:34.849636+0000", 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.53885770399999999, 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038135439999999999, 2024-09-16T06:48:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849636+0000", 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849636+0000", 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:35.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849634+0000", 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849671+0000", 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7101000000000003e-05 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849738+0000", 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6568000000000004e-05 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849752+0000", 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4309e-05 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849800+0000", 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8102999999999997e-05 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.849806+0000", 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6590000000000004e-06 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.853389+0000", 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035834339999999999 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.853449+0000", 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0086e-05 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.486 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:35.487 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:35.487 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:35.489 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 47, 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "b2976256-637c-4e28-94f6-99f20017bc5e", 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: "created": "2024-09-16T06:47:53.157733+0000", 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: "modified": "2024-09-16T06:48:34.808383+0000", 2024-09-16T06:48:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: "last_up_change": "2024-09-16T06:48:06.616578+0000", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "last_in_change": "2024-09-16T06:48:24.894403+0000", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_num": 5806080, 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_set": [ 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "nodeep-scrub", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "noscrub", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "pglog_hardlimit", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "purged_snapdirs", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_deletes", 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "sortbitwise" 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_version": 6, 2024-09-16T06:48:35.491 INFO:teuthology.orchestra.run.smithi100.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_snapshot": "", 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_max": 3, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd": 16, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "require_min_compat_client": "luminous", 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "min_compat_client": "jewel", 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "require_osd_release": "reef", 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_crimson": false, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pools": [ 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 1, 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": ".mgr", 2024-09-16T06:48:35.492 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:05.838641+0000", 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 1, 2024-09-16T06:48:35.493 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 1, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 1, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 1, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 1, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "25", 2024-09-16T06:48:35.494 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:35.495 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:35.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:35.497 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_max": 32, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 1 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": {} 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 15.380000114440918, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 15.380000114440918, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.12999999523162842, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-16T06:48:35.498 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "rbd", 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:08.489111+0000", 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:35.499 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 8, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 8, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 8, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 8, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 8, 2024-09-16T06:48:35.500 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "28", 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:35.501 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 3, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 28, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:35.502 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:35.503 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd": {} 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 4.0199999809265137, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 4.0199999809265137, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.56000000238418579, 2024-09-16T06:48:35.504 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2.25, 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2.25, 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.506 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:35.506 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "unique_pool_0", 2024-09-16T06:48:35.506 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:18.885917+0000", 2024-09-16T06:48:35.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:35.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:35.506 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 3, 2024-09-16T06:48:35.507 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 7, 2024-09-16T06:48:35.507 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 5, 2024-09-16T06:48:35.507 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 1, 2024-09-16T06:48:35.507 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:35.507 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:35.507 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 16, 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 16, 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 16, 2024-09-16T06:48:35.508 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 16, 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 16, 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:35.509 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "47", 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:35.510 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 13, 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 47, 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:35.511 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:35.512 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "shecprofile", 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:35.513 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:35.514 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 16384, 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:35.515 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: "rados": {} 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: "osds": [ 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:35.516 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "54494294-f25f-41f2-bff7-106492665e07", 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.517 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6808", 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6809", 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.518 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6810", 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6811", 2024-09-16T06:48:35.519 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6814", 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6815", 2024-09-16T06:48:35.520 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6812", 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6813", 2024-09-16T06:48:35.521 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6809/1028275949", 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6811/1028275949", 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6815/1028275949", 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6813/1028275949", 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "a265e752-1cd6-497e-bcb9-c56591a5ce30", 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.523 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6808", 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6809", 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.524 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6811", 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6813", 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.525 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6819", 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6821", 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.526 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6815", 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6817", 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.527 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6809/930331367", 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6813/930331367", 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6821/930331367", 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6817/930331367", 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "dada34c6-4dc6-4f9f-bc31-85983fd2d7fd", 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.528 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6800", 2024-09-16T06:48:35.529 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6801", 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6802", 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6803", 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.531 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6806", 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6807", 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6804", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6805", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6801/4231506893", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6803/4231506893", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6807/4231506893", 2024-09-16T06:48:35.533 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6805/4231506893", 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "6c0fc051-a4a0-4ce5-989a-668e75bbe3c0", 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.534 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6802", 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.535 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6803", 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6804", 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.536 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6805", 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6808", 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.537 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6809", 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6806", 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6807", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6803/1995864012", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6805/1995864012", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6809/1995864012", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6807/1995864012", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.539 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "36a46387-b859-4937-bbd0-5dd708630c6e", 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.540 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.541 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6802", 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6803", 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.542 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6806", 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6807", 2024-09-16T06:48:35.543 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6804", 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6805", 2024-09-16T06:48:35.544 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6801/2054405220", 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6803/2054405220", 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6807/2054405220", 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6805/2054405220", 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "073f3d23-77f0-4fd3-b067-27fa81137288", 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6810", 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6812", 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.547 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6814", 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6816", 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.548 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6822", 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6823", 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.549 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6818", 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6820", 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6812/1081990569", 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6816/1081990569", 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6823/1081990569", 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6820/1081990569", 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:35.551 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "8a81592f-3704-48f6-89ba-cc4997c2105e", 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.552 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6824", 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6825", 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.553 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6826", 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6827", 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.554 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6830", 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6831", 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6828", 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6829", 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6825/1095047955", 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6827/1095047955", 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6831/1095047955", 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6829/1095047955", 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27", 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.557 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6818", 2024-09-16T06:48:35.558 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6819", 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.559 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6820", 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6821", 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6824", 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6825", 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6822", 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6823", 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6819/3607859516", 2024-09-16T06:48:35.562 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6821/3607859516", 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6825/3607859516", 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6823/3607859516", 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7ce38131-057f-4d45-88a5-dae8bcceae3c", 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.563 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6816", 2024-09-16T06:48:35.564 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6817", 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6818", 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6819", 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6822", 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6823", 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.567 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6820", 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6821", 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.568 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6817/232055141", 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6819/232055141", 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6823/232055141", 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6821/232055141", 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "aa7fde6e-6563-4e37-8af2-9114d0091362", 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.569 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.570 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6824", 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6825", 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6826", 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6827", 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6830", 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6831", 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.573 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6828", 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6829", 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.574 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6825/1511881207", 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6827/1511881207", 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6831/1511881207", 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6829/1511881207", 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "05c5389d-8082-4be3-aa6d-138e00720f06", 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.575 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6808", 2024-09-16T06:48:35.576 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6809", 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.577 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6810", 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6811", 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.578 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6814", 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6815", 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6812", 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6813", 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6809/435900412", 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6811/435900412", 2024-09-16T06:48:35.580 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6815/435900412", 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6813/435900412", 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "99552590-1d0a-420e-b0c3-357246880658", 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.581 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6810", 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6811", 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6812", 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6813", 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6816", 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6817", 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6814", 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6815", 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6811/3208528083", 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6813/3208528083", 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6817/3208528083", 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6815/3208528083", 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.586 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "165fea1c-74b6-4806-a0d9-47441e161ed5", 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:35.587 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6824", 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6825", 2024-09-16T06:48:35.588 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6826", 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.589 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6827", 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6830", 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.590 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6831", 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6828", 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.591 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6829", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6825/3107910928", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6827/3107910928", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6831/3107910928", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6829/3107910928", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.592 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "03402d0c-e7fd-4233-b2cd-bf79cd75e9bc", 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 0, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 0, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:35.593 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6800", 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6801", 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.594 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6802", 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6803", 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.595 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6806", 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6807", 2024-09-16T06:48:35.596 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6804", 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.597 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6805", 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6801/316232366", 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6803/316232366", 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6807/316232366", 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6805/316232366", 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7b027faf-5c9e-4aee-9cb9-9c6459b8fe39", 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.599 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6816", 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6817", 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.600 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6818", 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6819", 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.601 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6822", 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6823", 2024-09-16T06:48:35.602 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6820", 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6821", 2024-09-16T06:48:35.603 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6817/693497656", 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6819/693497656", 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6823/693497656", 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6821/693497656", 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "4877aa40-c0d8-494f-8a98-36089ed694c2", 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:35.605 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6826", 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6827", 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.606 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6828", 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6829", 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6832", 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6833", 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.608 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6830", 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6831", 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:35.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6827/579912865", 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6829/579912865", 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6833/579912865", 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6831/579912865", 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.610 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_xinfo": [ 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.584963+0000", 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.611 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.677023+0000", 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.612 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.608639+0000", 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.384489+0000", 2024-09-16T06:48:35.613 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.480551+0000", 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.614 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.476690+0000", 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.615 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.615096+0000", 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.581224+0000", 2024-09-16T06:48:35.616 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.597357+0000", 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.549502+0000", 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.618 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.448485+0000", 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.619 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.495825+0000", 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.628407+0000", 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.620 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 65536, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.368452+0000", 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.621 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.600045+0000", 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.567594+0000", 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:35.622 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap": [], 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_items": [], 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_primaries": [], 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_temp": [], 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_temp": [], 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklist": {}, 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "range_blocklist": {}, 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profiles": { 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "default": { 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "2", 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "1", 2024-09-16T06:48:35.623 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "jerasure", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "reed_sol_van" 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "shecprofile": { 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "c": "2", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-device-class": "", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-root": "default", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "4", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "3", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "shecprofile", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "shec", 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "multiple" 2024-09-16T06:48:35.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps_queue": [ 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 2, 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:35.625 INFO:teuthology.orchestra.run.smithi100.stdout: "new_removed_snaps": [], 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "new_purged_snaps": [], 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_node_flags": {}, 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "device_class_flags": {}, 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode": { 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_enabled": false, 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_bucket_count": 0, 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "degraded_stretch_mode": 0, 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "recovering_stretch_mode": 0, 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_bucket": 0 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:35.626 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:35.627 INFO:tasks.thrashosds.thrasher:j is {'epoch': 47, 'fsid': 'b2976256-637c-4e28-94f6-99f20017bc5e', 'created': '2024-09-16T06:47:53.157733+0000', 'modified': '2024-09-16T06:48:34.808383+0000', 'last_up_change': '2024-09-16T06:48:06.616578+0000', 'last_in_change': '2024-09-16T06:48:24.894403+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T06:48:05.838641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T06:48:08.489111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-16T06:48:18.885917+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '54494294-f25f-41f2-bff7-106492665e07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6808', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6809', 'nonce': 1028275949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6810', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6811', 'nonce': 1028275949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6814', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6815', 'nonce': 1028275949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6812', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6813', 'nonce': 1028275949}]}, 'public_addr': '172.21.15.100:6809/1028275949', 'cluster_addr': '172.21.15.100:6811/1028275949', 'heartbeat_back_addr': '172.21.15.100:6815/1028275949', 'heartbeat_front_addr': '172.21.15.100:6813/1028275949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a265e752-1cd6-497e-bcb9-c56591a5ce30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 930331367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6811', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 930331367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6819', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 930331367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6815', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 930331367}]}, 'public_addr': '172.21.15.107:6809/930331367', 'cluster_addr': '172.21.15.107:6813/930331367', 'heartbeat_back_addr': '172.21.15.107:6821/930331367', 'heartbeat_front_addr': '172.21.15.107:6817/930331367', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'dada34c6-4dc6-4f9f-bc31-85983fd2d7fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 4231506893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 4231506893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 4231506893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 4231506893}]}, 'public_addr': '172.21.15.115:6801/4231506893', 'cluster_addr': '172.21.15.115:6803/4231506893', 'heartbeat_back_addr': '172.21.15.115:6807/4231506893', 'heartbeat_front_addr': '172.21.15.115:6805/4231506893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6c0fc051-a4a0-4ce5-989a-668e75bbe3c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6802', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6803', 'nonce': 1995864012}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6804', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6805', 'nonce': 1995864012}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6808', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6809', 'nonce': 1995864012}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6806', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6807', 'nonce': 1995864012}]}, 'public_addr': '172.21.15.117:6803/1995864012', 'cluster_addr': '172.21.15.117:6805/1995864012', 'heartbeat_back_addr': '172.21.15.117:6809/1995864012', 'heartbeat_front_addr': '172.21.15.117:6807/1995864012', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '36a46387-b859-4937-bbd0-5dd708630c6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 2054405220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6802', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6803', 'nonce': 2054405220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6806', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6807', 'nonce': 2054405220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6804', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6805', 'nonce': 2054405220}]}, 'public_addr': '172.21.15.100:6801/2054405220', 'cluster_addr': '172.21.15.100:6803/2054405220', 'heartbeat_back_addr': '172.21.15.100:6807/2054405220', 'heartbeat_front_addr': '172.21.15.100:6805/2054405220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '073f3d23-77f0-4fd3-b067-27fa81137288', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6812', 'nonce': 1081990569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6816', 'nonce': 1081990569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 1081990569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6820', 'nonce': 1081990569}]}, 'public_addr': '172.21.15.107:6812/1081990569', 'cluster_addr': '172.21.15.107:6816/1081990569', 'heartbeat_back_addr': '172.21.15.107:6823/1081990569', 'heartbeat_front_addr': '172.21.15.107:6820/1081990569', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8a81592f-3704-48f6-89ba-cc4997c2105e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 1095047955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1095047955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1095047955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1095047955}]}, 'public_addr': '172.21.15.115:6825/1095047955', 'cluster_addr': '172.21.15.115:6827/1095047955', 'heartbeat_back_addr': '172.21.15.115:6831/1095047955', 'heartbeat_front_addr': '172.21.15.115:6829/1095047955', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6818', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6819', 'nonce': 3607859516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6820', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6821', 'nonce': 3607859516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6824', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6825', 'nonce': 3607859516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6822', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6823', 'nonce': 3607859516}]}, 'public_addr': '172.21.15.117:6819/3607859516', 'cluster_addr': '172.21.15.117:6821/3607859516', 'heartbeat_back_addr': '172.21.15.117:6825/3607859516', 'heartbeat_front_addr': '172.21.15.117:6823/3607859516', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '7ce38131-057f-4d45-88a5-dae8bcceae3c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6816', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6817', 'nonce': 232055141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6818', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6819', 'nonce': 232055141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6822', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6823', 'nonce': 232055141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6820', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6821', 'nonce': 232055141}]}, 'public_addr': '172.21.15.100:6817/232055141', 'cluster_addr': '172.21.15.100:6819/232055141', 'heartbeat_back_addr': '172.21.15.100:6823/232055141', 'heartbeat_front_addr': '172.21.15.100:6821/232055141', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'aa7fde6e-6563-4e37-8af2-9114d0091362', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 1511881207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 1511881207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 1511881207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 1511881207}]}, 'public_addr': '172.21.15.107:6825/1511881207', 'cluster_addr': '172.21.15.107:6827/1511881207', 'heartbeat_back_addr': '172.21.15.107:6831/1511881207', 'heartbeat_front_addr': '172.21.15.107:6829/1511881207', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '05c5389d-8082-4be3-aa6d-138e00720f06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 435900412}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 435900412}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 435900412}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 435900412}]}, 'public_addr': '172.21.15.115:6809/435900412', 'cluster_addr': '172.21.15.115:6811/435900412', 'heartbeat_back_addr': '172.21.15.115:6815/435900412', 'heartbeat_front_addr': '172.21.15.115:6813/435900412', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '99552590-1d0a-420e-b0c3-357246880658', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6810', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6811', 'nonce': 3208528083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6812', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6813', 'nonce': 3208528083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6816', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6817', 'nonce': 3208528083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6814', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6815', 'nonce': 3208528083}]}, 'public_addr': '172.21.15.117:6811/3208528083', 'cluster_addr': '172.21.15.117:6813/3208528083', 'heartbeat_back_addr': '172.21.15.117:6817/3208528083', 'heartbeat_front_addr': '172.21.15.117:6815/3208528083', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '165fea1c-74b6-4806-a0d9-47441e161ed5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6824', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6825', 'nonce': 3107910928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6826', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6827', 'nonce': 3107910928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6830', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6831', 'nonce': 3107910928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6828', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6829', 'nonce': 3107910928}]}, 'public_addr': '172.21.15.100:6825/3107910928', 'cluster_addr': '172.21.15.100:6827/3107910928', 'heartbeat_back_addr': '172.21.15.100:6831/3107910928', 'heartbeat_front_addr': '172.21.15.100:6829/3107910928', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '03402d0c-e7fd-4233-b2cd-bf79cd75e9bc', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 316232366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 316232366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 316232366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 316232366}]}, 'public_addr': '172.21.15.107:6801/316232366', 'cluster_addr': '172.21.15.107:6803/316232366', 'heartbeat_back_addr': '172.21.15.107:6807/316232366', 'heartbeat_front_addr': '172.21.15.107:6805/316232366', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '7b027faf-5c9e-4aee-9cb9-9c6459b8fe39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 693497656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 693497656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 693497656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 693497656}]}, 'public_addr': '172.21.15.115:6817/693497656', 'cluster_addr': '172.21.15.115:6819/693497656', 'heartbeat_back_addr': '172.21.15.115:6823/693497656', 'heartbeat_front_addr': '172.21.15.115:6821/693497656', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '4877aa40-c0d8-494f-8a98-36089ed694c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6826', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6827', 'nonce': 579912865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6828', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6829', 'nonce': 579912865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 579912865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6830', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6831', 'nonce': 579912865}]}, 'public_addr': '172.21.15.117:6827/579912865', 'cluster_addr': '172.21.15.117:6829/579912865', 'heartbeat_back_addr': '172.21.15.117:6833/579912865', 'heartbeat_front_addr': '172.21.15.117:6831/579912865', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.584963+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.677023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.608639+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.384489+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.480551+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.476690+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.615096+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.581224+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.597357+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.549502+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.448485+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.495825+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.628407+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.368452+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.600045+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.567594+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T06:48:35.628 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T06:48:35.629 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:35.487+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:35.629 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:35.632+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:35.730 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:35.728+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:35.759 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:35.759 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:35.759 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:35.759 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:35.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:48:35.828 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:48:35.829 INFO:tasks.rados.rados.0.smithi117.stdout:128: done (3 left) 2024-09-16T06:48:35.829 INFO:tasks.rados.rados.0.smithi117.stdout:129: done (2 left) 2024-09-16T06:48:35.829 INFO:tasks.rados.rados.0.smithi117.stdout:133: done (1 left) 2024-09-16T06:48:35.829 INFO:tasks.rados.rados.0.smithi117.stdout:134: done (0 left) 2024-09-16T06:48:35.829 INFO:tasks.rados.rados.0.smithi117.stdout:135: snap_remove snap 4 2024-09-16T06:48:35.830 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:35.828+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:35.894 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:35.895 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:35.895 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:35.895 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:35.895 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:35.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:48:35.931 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:35.927+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:36.032 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:36.031+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:36.035 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=0, reqid=client.4511.0:1, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896059+0000", 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.141024495, 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034435659999999999, 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896059+0000", 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896059+0000", 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896057+0000", 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896147+0000", 2024-09-16T06:48:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9357000000000005e-05 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896273+0000", 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012586100000000001 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896297+0000", 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4173999999999999e-05 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896442+0000", 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000145625 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896478+0000", 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5880000000000002e-05 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899437+0000", 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002958792 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899503+0000", 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5959999999999999e-05 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/32 ECSubWrite(tid=0, reqid=client.4511.0:10, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898824+0000", 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.138259941999999, 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003449092, 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.040 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898824+0000", 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898824+0000", 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898823+0000", 2024-09-16T06:48:36.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898899+0000", 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5819e-05 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899029+0000", 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012984300000000001 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899053+0000", 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4400999999999999e-05 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899184+0000", 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000130443 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899205+0000", 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1744999999999999e-05 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902216+0000", 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030107409999999999 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902273+0000", 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6969999999999998e-05 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.044 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.13743571, 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064369220000000003, 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899647+0000", 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899689+0000", 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2429000000000003e-05 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899768+0000", 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9277999999999995e-05 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899797+0000", 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8473999999999999e-05 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903489+0000", 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036918680000000001 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903519+0000", 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0283000000000001e-05 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906024+0000", 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025045430000000001 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906085+0000", 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1637000000000006e-05 2024-09-16T06:48:36.048 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=2, reqid=client.4511.0:34, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.917931+0000", 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.119153133999999, 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004345916, 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917931+0000", 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917931+0000", 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917930+0000", 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918094+0000", 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016369 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918269+0000", 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000174935 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918294+0000", 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5250999999999999e-05 2024-09-16T06:48:36.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918470+0000", 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000176081 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918495+0000", 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4828e-05 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922227+0000", 2024-09-16T06:48:36.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037319929999999999 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922277+0000", 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9914999999999999e-05 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=3, reqid=client.4511.0:35, at_version=35'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.918102+0000", 2024-09-16T06:48:36.053 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.118981776, 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042176770000000004, 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918102+0000", 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918102+0000", 2024-09-16T06:48:36.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918101+0000", 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918128+0000", 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.639e-05 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918343+0000", 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021569499999999999 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918362+0000", 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814999999999999e-05 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919512+0000", 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011494210000000001 2024-09-16T06:48:36.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919534+0000", 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2452000000000001e-05 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922287+0000", 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027528869999999999 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922320+0000", 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2863e-05 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=18, reqid=client.4511.0:130, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.991510711, 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044945139999999998, 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045573+0000", 2024-09-16T06:48:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046385+0000", 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00081262499999999998 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046429+0000", 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3606000000000002e-05 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046441+0000", 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2021000000000001e-05 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046556+0000", 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114869 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046571+0000", 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5214e-05 2024-09-16T06:48:36.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050032+0000", 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00346102 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050068+0000", 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5651000000000001e-05 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=19, reqid=client.4511.0:131, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.046631+0000", 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.990453154000001, 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038867540000000001, 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046631+0000", 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046631+0000", 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046630+0000", 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046655+0000", 2024-09-16T06:48:36.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4590999999999999e-05 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046704+0000", 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9693e-05 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046720+0000", 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5415999999999999e-05 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047254+0000", 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00053385699999999999 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047262+0000", 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7569999999999993e-06 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050474+0000", 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032111470000000001 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050518+0000", 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4039000000000003e-05 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.988268680999999, 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052817840000000003, 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.067 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048832+0000", 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7638e-05 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048870+0000", 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8114000000000001e-05 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048877+0000", 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4849999999999997e-06 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.049092+0000", 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000214806 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.049100+0000", 2024-09-16T06:48:36.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8170000000000005e-06 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054030+0000", 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049292750000000003 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054097+0000", 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7453000000000006e-05 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=10, reqid=client.4511.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.985916515, 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005244037, 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051166+0000", 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051182+0000", 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5571000000000001e-05 2024-09-16T06:48:36.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051230+0000", 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7703000000000001e-05 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051243+0000", 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3573e-05 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051327+0000", 2024-09-16T06:48:36.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4098999999999996e-05 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051343+0000", 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5812e-05 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056375+0000", 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050313950000000001 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056411+0000", 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6890000000000001e-05 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=20, reqid=client.4511.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051662+0000", 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.985422413, 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047138700000000002, 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051662+0000", 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051662+0000", 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051660+0000", 2024-09-16T06:48:36.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051678+0000", 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7470999999999999e-05 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051735+0000", 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7170000000000003e-05 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051748+0000", 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2884999999999999e-05 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051827+0000", 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9423000000000002e-05 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051842+0000", 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4385e-05 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056338+0000", 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044967109999999996 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056375+0000", 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6955000000000001e-05 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.080 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=15, reqid=client.4511.0:133, at_version=35'9, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.052599+0000", 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.984485119, 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00380971, 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052599+0000", 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052599+0000", 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052598+0000", 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052728+0000", 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000129868 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052781+0000", 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3672999999999999e-05 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052792+0000", 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0312000000000001e-05 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052869+0000", 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7670999999999995e-05 2024-09-16T06:48:36.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.052895+0000", 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5829999999999998e-05 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056381+0000", 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034860619999999998 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056409+0000", 2024-09-16T06:48:36.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7381000000000001e-05 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 35/32 ECSubWrite(tid=16, reqid=client.4511.0:134, at_version=35'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.054246+0000", 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.982838146000001, 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003580537, 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054246+0000", 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054246+0000", 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054245+0000", 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054265+0000", 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9826e-05 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054329+0000", 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4335e-05 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054342+0000", 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3512999999999999e-05 2024-09-16T06:48:36.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054433+0000", 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0084e-05 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054449+0000", 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6393e-05 2024-09-16T06:48:36.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057786+0000", 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033369269999999999 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057826+0000", 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0460000000000002e-05 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=29, reqid=client.4511.0:159, at_version=35'7, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.512622+0000", 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.524462375000001, 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047019150000000001, 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.512622+0000", 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.512622+0000", 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.512620+0000", 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513065+0000", 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044465900000000003 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513171+0000", 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010642 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513195+0000", 2024-09-16T06:48:36.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3587e-05 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513371+0000", 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017610999999999999 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.513395+0000", 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4283000000000001e-05 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.517253+0000", 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038581560000000002 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.517324+0000", 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0359999999999997e-05 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=30, reqid=client.4511.0:160, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.522435157, 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056229460000000002, 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514647+0000", 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515002+0000", 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000354607 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515104+0000", 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102607 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515128+0000", 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3998e-05 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515678+0000", 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055015800000000003 2024-09-16T06:48:36.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515703+0000", 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4706999999999999e-05 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520193+0000", 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044896160000000001 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520272+0000", 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9089000000000001e-05 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=31, reqid=client.4511.0:161, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.521906153, 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056966849999999999, 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515177+0000", 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515216+0000", 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9765999999999998e-05 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515642+0000", 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042582399999999999 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515672+0000", 2024-09-16T06:48:36.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9403e-05 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516695+0000", 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010230549999999999 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516716+0000", 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1858e-05 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520824+0000", 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041076059999999998 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520875+0000", 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0479000000000002e-05 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 40/37 ECSubWrite(tid=43, reqid=client.4511.0:242, at_version=0'0, trim_to=0'0, roll_forward_to=40'18))", 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.989646+0000", 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.0474376620000001, 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040440190000000003, 2024-09-16T06:48:36.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989646+0000", 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989646+0000", 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989645+0000", 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989675+0000", 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0626000000000001e-05 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989753+0000", 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7935999999999994e-05 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989772+0000", 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8936e-05 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989878+0000", 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105721 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989892+0000", 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3562e-05 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993648+0000", 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003756747 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993690+0000", 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2045000000000001e-05 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.107 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 40/37 ECSubWrite(tid=59, reqid=client.4511.0:233, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.0427612740000001, 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061889309999999999, 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994322+0000", 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995376+0000", 2024-09-16T06:48:36.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00105338 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995464+0000", 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8170000000000005e-05 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995483+0000", 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9548999999999999e-05 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995633+0000", 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014933200000000001 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995662+0000", 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8847999999999999e-05 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000463+0000", 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048011360000000001 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000512+0000", 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8860999999999999e-05 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 42/32 ECSubWrite(tid=28, reqid=client.4511.0:292, at_version=42'11, trim_to=0'0, roll_forward_to=35'7))", 2024-09-16T06:48:36.112 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.108643+0000", 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9284414319999996, 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037528240000000001, 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108643+0000", 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108643+0000", 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108641+0000", 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108682+0000", 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1717000000000001e-05 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108788+0000", 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105794 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108810+0000", 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1625e-05 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108951+0000", 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000141392 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108979+0000", 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7678000000000002e-05 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.112337+0000", 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033586290000000001 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.112395+0000", 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7951999999999997e-05 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.116 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 47/37 ECSubWrite(tid=91, reqid=client.4511.0:355, at_version=47'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.1904406139999999, 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061968889999999997, 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846717+0000", 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4350999999999993e-05 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846862+0000", 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014473599999999999 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846886+0000", 2024-09-16T06:48:36.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4289000000000001e-05 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.847145+0000", 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000259158 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.847176+0000", 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1154e-05 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.852788+0000", 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056113070000000003 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.852840+0000", 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2509999999999997e-05 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 47/37 ECSubWrite(tid=92, reqid=client.4511.0:356, at_version=47'23, trim_to=0'0, roll_forward_to=44'20))", 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:34.850020+0000", 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.1870639629999999, 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003670244, 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850020+0000", 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850020+0000", 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850018+0000", 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850068+0000", 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9669000000000002e-05 2024-09-16T06:48:36.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850148+0000", 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0397999999999998e-05 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850170+0000", 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1778999999999999e-05 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850290+0000", 2024-09-16T06:48:36.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011989099999999999 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.850314+0000", 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3728000000000001e-05 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.853613+0000", 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032996240000000001 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.853690+0000", 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6851e-05 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.126 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:36.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:48:36.133 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:36.132+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:36.234 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:36.236+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:36.296 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:36.296 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:36.297 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:36.297 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:36.310 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:48:36.334 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:36.332+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:36.427 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:36.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:36.427 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:36.427 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:36.427 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:36.435 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:36.432+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:36.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:48:36.536 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:36.536+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:36.562 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:7 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.666363243999999, 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023918795999999999, 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895875+0000", 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896153+0000", 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000277688 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896194+0000", 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1916000000000003e-05 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896222+0000", 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.726e-05 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896390+0000", 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016809299999999999 2024-09-16T06:48:36.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896750+0000", 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036068999999999999 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898496+0000", 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017459070000000001 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906280+0000", 2024-09-16T06:48:36.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077831310000000004 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919636+0000", 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013356191 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919798+0000", 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016195000000000001 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:8 3.1s0 3:825f7e25:::smithi11731613-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.665712817999999, 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031459483000000003, 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 8 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896527+0000", 2024-09-16T06:48:36.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896557+0000", 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9323999999999999e-05 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896578+0000", 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1038000000000001e-05 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896594+0000", 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6038000000000001e-05 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900822+0000", 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042283219999999996 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901041+0000", 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021897799999999999 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902173+0000", 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00113226 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911105+0000", 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089313959999999994 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927760+0000", 2024-09-16T06:48:36.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016655313000000001 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927989+0000", 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022873699999999999 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:9 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.665631789000001, 2024-09-16T06:48:36.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.040864080999999997, 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 9 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896609+0000", 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896617+0000", 2024-09-16T06:48:36.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9589999999999995e-06 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896634+0000", 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6872e-05 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896647+0000", 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3621999999999999e-05 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902794+0000", 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061462820000000003 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903015+0000", 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221487 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928840+0000", 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025824686999999999 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929046+0000", 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020604299999999999 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.937474+0000", 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084285460000000003 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.579 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:16 3.bs0 3:db3fccb6:::smithi11731613-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.665582177999999, 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024707216000000001, 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896659+0000", 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897043+0000", 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038430400000000003 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897068+0000", 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5417000000000001e-05 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897087+0000", 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8218000000000001e-05 2024-09-16T06:48:36.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897200+0000", 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011297499999999999 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897502+0000", 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030200600000000001 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899657+0000", 2024-09-16T06:48:36.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021558250000000001 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911012+0000", 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011354401 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921200+0000", 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010188033000000001 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921367+0000", 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016735 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:17 3.bs0 3:db3fccb6:::smithi11731613-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.664355151000001, 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034791753000000002, 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 17 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897885+0000", 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897900+0000", 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5515e-05 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897935+0000", 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4703000000000003e-05 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897965+0000", 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9884000000000001e-05 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905007+0000", 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007042296 2024-09-16T06:48:36.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905241+0000", 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023388 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014152209999999999 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912065+0000", 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054085640000000003 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932510+0000", 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020444922000000001 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932679+0000", 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168894 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:18 3.bs0 3:db3fccb6:::smithi11731613-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.664196614, 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.041379560000000003, 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 18 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.591 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898044+0000", 2024-09-16T06:48:36.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898055+0000", 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0902e-05 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898077+0000", 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2395000000000002e-05 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898095+0000", 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7825e-05 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907329+0000", 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092342250000000004 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907544+0000", 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021447400000000001 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932831+0000", 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025286896999999999 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933015+0000", 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018463 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939425+0000", 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064098250000000001 2024-09-16T06:48:36.595 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:37 3.bs0 3:d4957fab:::smithi11731613-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.65705741, 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029935371999999998, 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.596 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 37 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905182+0000", 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905624+0000", 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044199799999999998 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905656+0000", 2024-09-16T06:48:36.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2393e-05 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905676+0000", 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0285e-05 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907630+0000", 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019531459999999998 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907646+0000", 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6628000000000001e-05 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914043+0000", 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063962699999999999 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914333+0000", 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029089300000000002 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916429+0000", 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002095085 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923501+0000", 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007072404 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934947+0000", 2024-09-16T06:48:36.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011445688000000001 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.935120+0000", 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017347200000000001 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:38 3.bs0 3:d4957fab:::smithi11731613-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.656544923, 2024-09-16T06:48:36.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034120408999999997, 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 38 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905696+0000", 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905706+0000", 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9890000000000004e-06 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905725+0000", 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9341e-05 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905739+0000", 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4336e-05 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907725+0000", 2024-09-16T06:48:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001985849 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907741+0000", 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5401e-05 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918115+0000", 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010374431999999999 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918349+0000", 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023440900000000001 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919836+0000", 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014862639999999999 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.925093+0000", 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052577300000000004 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939675+0000", 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014581904999999999 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939818+0000", 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014229600000000001 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:39 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.608 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.656488795, 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039069524000000001, 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 39 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:48:36.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905752+0000", 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905759+0000", 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905776+0000", 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6694e-05 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905788+0000", 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2336e-05 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907813+0000", 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020251790000000002 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907828+0000", 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5094e-05 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921873+0000", 2024-09-16T06:48:36.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014044235 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922092+0000", 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000219514 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940712+0000", 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018619988000000001 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940891+0000", 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017858199999999999 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.944823+0000", 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039321989999999999 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:40 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.614 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.65644088, 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027254417, 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 40 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:48:36.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905800+0000", 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906496+0000", 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00069684800000000002 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906524+0000", 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7344000000000001e-05 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906543+0000", 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9468000000000001e-05 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906717+0000", 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000173692 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906758+0000", 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1186000000000002e-05 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.913762+0000", 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070039339999999999 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914055+0000", 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029257000000000003 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916068+0000", 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020128730000000001 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921559+0000", 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054913260000000004 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932938+0000", 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011379031 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933056+0000", 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117785 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:41 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.65566488, 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027013423000000002, 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 41 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906575+0000", 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906603+0000", 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8030000000000001e-05 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906622+0000", 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9008000000000002e-05 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906638+0000", 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5469e-05 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906985+0000", 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034709699999999999 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906998+0000", 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3512e-05 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917405+0000", 2024-09-16T06:48:36.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010406425 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917725+0000", 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032018999999999998 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919697+0000", 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019721069999999999 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924833+0000", 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051359170000000003 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933307+0000", 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084737270000000003 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933591+0000", 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028407699999999998 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:42 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.655586253999999, 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031605503, 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 42 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906654+0000", 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906661+0000", 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5719999999999999e-06 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906678+0000", 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7059e-05 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906698+0000", 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9551e-05 2024-09-16T06:48:36.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907075+0000", 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037732500000000002 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907087+0000", 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1745e-05 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920576+0000", 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013489655 2024-09-16T06:48:36.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920843+0000", 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026655899999999998 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934288+0000", 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013445297 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934560+0000", 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000272238 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.938261+0000", 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037009289999999999 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/32 ECSubWrite(tid=2, reqid=client.4511.0:19, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:36.632 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.655380625999999, 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018027405, 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:36.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907090+0000", 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229569 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919015+0000", 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011924924999999999 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919062+0000", 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6316000000000002e-05 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919376+0000", 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031417700000000001 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919401+0000", 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4854999999999998e-05 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924838+0000", 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054377970000000003 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924889+0000", 2024-09-16T06:48:36.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0445000000000002e-05 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:105 3.6s0 3:64026cd9:::smithi11731613-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.555658684000001, 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023735583000000001, 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.637 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 105 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006587+0000", 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6360000000000002e-06 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006598+0000", 2024-09-16T06:48:36.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1151999999999999e-05 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006608+0000", 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0036e-05 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.011103+0000", 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044949100000000004 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.011267+0000", 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000163778 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018815+0000", 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007547705 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018930+0000", 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115093 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.030319+0000", 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011389132999999999 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.642 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:108 3.6s0 3:69f2a009:::smithi11731613-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.551916905000001, 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021014327999999999, 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 108 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010324+0000", 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010330+0000", 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4829999999999996e-06 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010348+0000", 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7878000000000001e-05 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010365+0000", 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6770999999999999e-05 2024-09-16T06:48:36.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020964+0000", 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010598849 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021064+0000", 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100537 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028570+0000", 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075053480000000002 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028742+0000", 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017269299999999999 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.031340+0000", 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002597168 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.647 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:111 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.548466854000001, 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019128665999999999, 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 111 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.648 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013774+0000", 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013781+0000", 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7320000000000001e-06 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013798+0000", 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7665000000000001e-05 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013812+0000", 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3745e-05 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018077+0000", 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004264561 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018218+0000", 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014137000000000001 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.023743+0000", 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055253430000000003 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.023904+0000", 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016036100000000001 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.032904+0000", 2024-09-16T06:48:36.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090001990000000004 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:113 3.1s0 3:87a9bcdd:::smithi11731613-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.544405701000001, 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017992622999999999, 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 113 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:48:36.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017835+0000", 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017845+0000", 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9189999999999994e-06 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017876+0000", 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1446999999999999e-05 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017889+0000", 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2571e-05 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020765+0000", 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028765259999999999 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020877+0000", 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111523 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021333+0000", 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00045663300000000002 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.022097+0000", 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00076327800000000005 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035682+0000", 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013585000999999999 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035829+0000", 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000147367 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.658 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:114 3.1s0 3:87a9bcdd:::smithi11731613-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.544341936, 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021941939000000001, 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 114 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017899+0000", 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017905+0000", 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7209999999999999e-06 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017917+0000", 2024-09-16T06:48:36.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1979e-05 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017926+0000", 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1430000000000004e-06 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021590+0000", 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036640029999999999 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021682+0000", 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2441000000000005e-05 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036527+0000", 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014844758 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036729+0000", 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020231099999999999 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.039842+0000", 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031126779999999998 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:152 3.7s0 3:ebfb55a4:::smithi11731613-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.485834707, 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44514418, 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 152 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076406+0000", 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076414+0000", 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2889999999999998e-06 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076425+0000", 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0671e-05 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076433+0000", 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0870000000000007e-06 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079120+0000", 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026864800000000002 2024-09-16T06:48:36.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079194+0000", 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4874000000000002e-05 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079523+0000", 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032904499999999998 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080194+0000", 2024-09-16T06:48:36.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067098400000000001 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521344+0000", 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.441149183 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521552+0000", 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020799600000000001 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:153 3.7s0 3:ebfb55a4:::smithi11731613-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.485800880999999, 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44943085999999999, 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 153 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:36.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076440+0000", 2024-09-16T06:48:36.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076444+0000", 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0559999999999998e-06 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076455+0000", 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0789e-05 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076466+0000", 2024-09-16T06:48:36.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0241000000000001e-05 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080491+0000", 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040256060000000001 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080588+0000", 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6849999999999996e-05 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:36.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522277+0000", 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44168930299999998 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522519+0000", 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241246 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.525872+0000", 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033535660000000001 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:36.676 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:36.676 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:36.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:36.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:48:36.678 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:36.675+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:36.778 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:36.775+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:36.810 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:36.810 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:36.825 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:48:36.830 INFO:tasks.rados.rados.0.smithi117.stdout:135: done (0 left) 2024-09-16T06:48:36.830 INFO:tasks.rados.rados.0.smithi117.stdout:136: delete oid 21 current snap is 11 2024-09-16T06:48:36.846 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:36.846 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:36.850 INFO:tasks.rados.rados.0.smithi117.stdout:136: done (0 left) 2024-09-16T06:48:36.850 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 32 current snap is 11 2024-09-16T06:48:36.850 INFO:tasks.rados.rados.0.smithi117.stdout:137: seq_num 77 ranges {524288=524288,1048576=524288,1572864=245760} 2024-09-16T06:48:36.871 INFO:tasks.rados.rados.0.smithi117.stdout:137: writing smithi11731613-32 from 524288 to 1048576 tid 1 2024-09-16T06:48:36.872 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:36.873 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:36.879 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:36.879+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:36.883 INFO:tasks.rados.rados.0.smithi117.stdout:137: writing smithi11731613-32 from 1048576 to 1572864 tid 2 2024-09-16T06:48:36.888 INFO:tasks.rados.rados.0.smithi117.stdout:137: writing smithi11731613-32 from 1572864 to 1818624 tid 3 2024-09-16T06:48:36.888 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 10 current snap is 11 2024-09-16T06:48:36.889 INFO:tasks.rados.rados.0.smithi117.stdout:138: seq_num 78 ranges {1081344=557056,1638400=671744,2310144=98304} 2024-09-16T06:48:36.894 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:36.894 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:36.909 INFO:tasks.rados.rados.0.smithi117.stdout:138: writing smithi11731613-10 from 1081344 to 1638400 tid 1 2024-09-16T06:48:36.917 INFO:tasks.rados.rados.0.smithi117.stdout:138: writing smithi11731613-10 from 1638400 to 2310144 tid 2 2024-09-16T06:48:36.918 INFO:tasks.rados.rados.0.smithi117.stdout:138: writing smithi11731613-10 from 2310144 to 2408448 tid 3 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:139: rollback oid 15 current snap is 11 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 15 to 9 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:137: finishing write tid 1 to smithi11731613-32 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:137: finishing write tid 2 to smithi11731613-32 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:137: finishing write tid 3 to smithi11731613-32 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:137: finishing write tid 4 to smithi11731613-32 2024-09-16T06:48:36.919 INFO:tasks.rados.rados.0.smithi117.stdout:140: snap_remove snap 6 2024-09-16T06:48:36.919 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:36.920 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:36.941 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:36.941 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:36.965 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:36.965 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:36.980 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:36.980+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:36.988 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:36.988 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:36.995 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:36.996 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:48:37.016 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:37.016 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:37.042 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:37.043 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:37.072 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:37.073 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:37.081 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:37.080+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:37.096 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:37.096 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:37.132 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:37.132 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:48:37.181 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:37.180+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:37.270 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.282 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:37.279+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:37.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:48:37.330 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:37.330 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:37.363 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:37.363 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:37.384 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:37.380+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:37.401 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:37.401 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:37.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:37.408+0000 7f93927fc640 0 -- 172.21.15.100:0/2846126801 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7f939421e610 msgr2=0x7f9394220a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:37.444 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:37.444 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:37.451 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:37.451 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:48:37.475 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:37.475 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:37.484 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:37.480+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:37.585 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:37.587+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:37.587 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:37.587 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:48:37.686 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:37.687+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:37.719 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:37.719 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:48:37.787 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:37.787+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:140: done (3 left) 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 16 current snap is 11 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:138: finishing write tid 1 to smithi11731613-10 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:139: finishing rollback tid 0 to smithi11731613-15 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:137: finishing write tid 5 to smithi11731613-32 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:137: oid 32 updating version 0 to 53 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:137: oid 32 updating version 53 to 55 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:137: oid 32 updating version 55 to 56 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:137: oid 32 version 56 is already newer than 54 2024-09-16T06:48:37.835 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 32 v 56 (ObjNum 77 snap 11 seq_num 77) dirty exists 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:137: left oid 32 (ObjNum 77 snap 11 seq_num 77) 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: finishing write tid 2 to smithi11731613-10 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:139: finishing rollback tid 1 to smithi11731613-15 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:139: finishing rollback tid 2 to smithi11731613-15 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 15 v 46 (ObjNum 72 snap 6 seq_num 72) dirty exists 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: finishing write tid 3 to smithi11731613-10 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: finishing write tid 4 to smithi11731613-10 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: finishing write tid 5 to smithi11731613-10 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: oid 10 updating version 0 to 61 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: oid 10 version 61 is already newer than 60 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: oid 10 version 61 is already newer than 59 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: oid 10 version 61 is already newer than 58 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 61 (ObjNum 78 snap 11 seq_num 78) dirty exists 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:138: left oid 10 (ObjNum 78 snap 11 seq_num 78) 2024-09-16T06:48:37.836 INFO:tasks.rados.rados.0.smithi117.stdout:141: seq_num 79 ranges {262144=573440,835584=425984,1261568=344064} 2024-09-16T06:48:37.849 INFO:tasks.rados.rados.0.smithi117.stdout:141: writing smithi11731613-16 from 262144 to 835584 tid 1 2024-09-16T06:48:37.855 INFO:tasks.rados.rados.0.smithi117.stdout:141: writing smithi11731613-16 from 835584 to 1261568 tid 2 2024-09-16T06:48:37.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:48:37.859 INFO:tasks.rados.rados.0.smithi117.stdout:141: writing smithi11731613-16 from 1261568 to 1605632 tid 3 2024-09-16T06:48:37.860 INFO:tasks.rados.rados.0.smithi117.stdout:137: done (3 left) 2024-09-16T06:48:37.860 INFO:tasks.rados.rados.0.smithi117.stdout:138: done (2 left) 2024-09-16T06:48:37.860 INFO:tasks.rados.rados.0.smithi117.stdout:139: done (1 left) 2024-09-16T06:48:37.860 INFO:tasks.rados.rados.0.smithi117.stdout:142: setattr oid 37 current snap is 11 2024-09-16T06:48:37.865 INFO:tasks.rados.rados.0.smithi117.stdout:143: copy_from oid 31 from oid 41 current snap is 11 2024-09-16T06:48:37.865 INFO:tasks.rados.rados.0.smithi117.stdout:144: setattr oid 45 current snap is 11 2024-09-16T06:48:37.869 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 47 current snap is 11 2024-09-16T06:48:37.869 INFO:tasks.rados.rados.0.smithi117.stdout:145: seq_num 80 ranges {1671168=589824,2260992=573440,2834432=196608} 2024-09-16T06:48:37.870 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:37.870 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:37.887 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:37.884+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:37.888 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:48:37.891 INFO:tasks.rados.rados.0.smithi117.stdout:145: writing smithi11731613-47 from 1671168 to 2260992 tid 1 2024-09-16T06:48:37.898 INFO:tasks.rados.rados.0.smithi117.stdout:145: writing smithi11731613-47 from 2260992 to 2834432 tid 2 2024-09-16T06:48:37.900 INFO:tasks.rados.rados.0.smithi117.stdout:145: writing smithi11731613-47 from 2834432 to 3031040 tid 3 2024-09-16T06:48:37.900 INFO:tasks.rados.rados.0.smithi117.stdout:146: read oid 36 snap -1 2024-09-16T06:48:37.900 INFO:tasks.rados.rados.0.smithi117.stdout:146: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:48:37.900 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 19 current snap is 11 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: finishing write tid 1 to smithi11731613-16 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: finishing write tid 2 to smithi11731613-16 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 19 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: finishing write tid 3 to smithi11731613-16 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: finishing write tid 4 to smithi11731613-16 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 45 v 34 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: finishing write tid 5 to smithi11731613-16 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: oid 16 updating version 0 to 23 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: oid 16 version 23 is already newer than 22 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: oid 16 version 23 is already newer than 21 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: oid 16 updating version 23 to 24 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 16 v 24 (ObjNum 79 snap 11 seq_num 79) dirty exists 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:141: left oid 16 (ObjNum 79 snap 11 seq_num 79) 2024-09-16T06:48:37.901 INFO:tasks.rados.rados.0.smithi117.stdout:147: seq_num 81 ranges {311296=606208,917504=720896,1638400=49152} 2024-09-16T06:48:37.910 INFO:tasks.rados.rados.0.smithi117.stdout:147: writing smithi11731613-19 from 311296 to 917504 tid 1 2024-09-16T06:48:37.918 INFO:tasks.rados.rados.0.smithi117.stdout:147: writing smithi11731613-19 from 917504 to 1638400 tid 2 2024-09-16T06:48:37.918 INFO:tasks.rados.rados.0.smithi117.stdout:147: writing smithi11731613-19 from 1638400 to 1687552 tid 3 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:141: done (6 left) 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:142: done (5 left) 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:144: done (4 left) 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:148: read oid 12 snap 9 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:148: expect deleted 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:149: rollback oid 17 current snap is 11 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 17 to 3 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 42 current snap is 11 2024-09-16T06:48:37.919 INFO:tasks.rados.rados.0.smithi117.stdout:150: seq_num 82 ranges {278528=622592,901120=475136,1376256=294912} 2024-09-16T06:48:37.927 INFO:tasks.rados.rados.0.smithi117.stdout:150: writing smithi11731613-42 from 278528 to 901120 tid 1 2024-09-16T06:48:37.932 INFO:tasks.rados.rados.0.smithi117.stdout:150: writing smithi11731613-42 from 901120 to 1376256 tid 2 2024-09-16T06:48:37.935 INFO:tasks.rados.rados.0.smithi117.stdout:150: writing smithi11731613-42 from 1376256 to 1671168 tid 3 2024-09-16T06:48:37.935 INFO:tasks.rados.rados.0.smithi117.stdout:151: read oid 37 snap 5 2024-09-16T06:48:37.935 INFO:tasks.rados.rados.0.smithi117.stdout:151: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-16T06:48:37.935 INFO:tasks.rados.rados.0.smithi117.stdout:152: delete oid 22 current snap is 11 2024-09-16T06:48:37.935 INFO:tasks.rados.rados.0.smithi117.stdout:146: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:48:37.980 INFO:tasks.rados.rados.0.smithi117.stdout:145: finishing write tid 1 to smithi11731613-47 2024-09-16T06:48:37.980 INFO:tasks.rados.rados.0.smithi117.stdout:145: finishing write tid 2 to smithi11731613-47 2024-09-16T06:48:37.980 INFO:tasks.rados.rados.0.smithi117.stdout:145: finishing write tid 3 to smithi11731613-47 2024-09-16T06:48:37.980 INFO:tasks.rados.rados.0.smithi117.stdout:147: finishing write tid 1 to smithi11731613-19 2024-09-16T06:48:37.980 INFO:tasks.rados.rados.0.smithi117.stdout:149: finishing rollback tid 0 to smithi11731613-17 2024-09-16T06:48:37.980 INFO:tasks.rados.rados.0.smithi117.stdout:147: finishing write tid 2 to smithi11731613-19 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:147: finishing write tid 3 to smithi11731613-19 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:143: finishing copy_from racing read to smithi11731613-31 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:149: finishing rollback tid 1 to smithi11731613-17 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:149: finishing rollback tid 2 to smithi11731613-17 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 17 v 26 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:150: finishing write tid 1 to smithi11731613-42 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:150: finishing write tid 2 to smithi11731613-42 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:143: finishing copy_from to smithi11731613-31 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 31 v 42 (ObjNum 73 snap 7 seq_num 73) dirty exists 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:150: finishing write tid 3 to smithi11731613-42 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:150: finishing write tid 4 to smithi11731613-42 2024-09-16T06:48:37.981 INFO:tasks.rados.rados.0.smithi117.stdout:151: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-16T06:48:37.988 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:37.983+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:150: finishing write tid 5 to smithi11731613-42 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:150: oid 42 updating version 0 to 29 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:150: oid 42 version 29 is already newer than 28 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:150: oid 42 updating version 29 to 31 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:150: oid 42 version 31 is already newer than 30 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 42 v 31 (ObjNum 82 snap 11 seq_num 82) dirty exists 2024-09-16T06:48:38.008 INFO:tasks.rados.rados.0.smithi117.stdout:150: left oid 42 (ObjNum 82 snap 11 seq_num 82) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:143: done (8 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:146: done (7 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:148: done (6 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:149: done (5 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:150: done (4 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:151: done (3 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:152: done (2 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:153: rmattr oid 31 current snap is 11 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:153: done (2 left) 2024-09-16T06:48:38.009 INFO:tasks.rados.rados.0.smithi117.stdout:154: setattr oid 33 current snap is 11 2024-09-16T06:48:38.013 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.013 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.013 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 17 current snap is 11 2024-09-16T06:48:38.013 INFO:tasks.rados.rados.0.smithi117.stdout:155: seq_num 84 ranges {278528=655360,933888=638976,1572864=131072} 2024-09-16T06:48:38.028 INFO:tasks.rados.rados.0.smithi117.stdout:155: writing smithi11731613-17 from 278528 to 933888 tid 1 2024-09-16T06:48:38.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:48:38.039 INFO:tasks.rados.rados.0.smithi117.stdout:155: writing smithi11731613-17 from 933888 to 1572864 tid 2 2024-09-16T06:48:38.041 INFO:tasks.rados.rados.0.smithi117.stdout:155: writing smithi11731613-17 from 1572864 to 1703936 tid 3 2024-09-16T06:48:38.041 INFO:tasks.rados.rados.0.smithi117.stdout:156: snap_create 2024-09-16T06:48:38.041 INFO:tasks.rados.rados.0.smithi117.stdout:147: finishing write tid 4 to smithi11731613-19 2024-09-16T06:48:38.041 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 33 v 47 (ObjNum 74 snap 10 seq_num 74) dirty exists 2024-09-16T06:48:38.041 INFO:tasks.rados.rados.0.smithi117.stdout:147: finishing write tid 5 to smithi11731613-19 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:147: oid 19 updating version 0 to 43 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:147: oid 19 updating version 43 to 44 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:147: oid 19 updating version 44 to 45 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:147: oid 19 version 45 is already newer than 42 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 19 v 45 (ObjNum 81 snap 11 seq_num 81) dirty exists 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:147: left oid 19 (ObjNum 81 snap 11 seq_num 81) 2024-09-16T06:48:38.042 INFO:tasks.rados.rados.0.smithi117.stdout:155: finishing write tid 1 to smithi11731613-17 2024-09-16T06:48:38.046 INFO:tasks.rados.rados.0.smithi117.stdout:155: finishing write tid 2 to smithi11731613-17 2024-09-16T06:48:38.047 INFO:tasks.rados.rados.0.smithi117.stdout:155: finishing write tid 3 to smithi11731613-17 2024-09-16T06:48:38.051 INFO:tasks.rados.rados.0.smithi117.stdout:155: finishing write tid 4 to smithi11731613-17 2024-09-16T06:48:38.054 INFO:tasks.rados.rados.0.smithi117.stdout:155: finishing write tid 5 to smithi11731613-17 2024-09-16T06:48:38.054 INFO:tasks.rados.rados.0.smithi117.stdout:155: oid 17 updating version 0 to 34 2024-09-16T06:48:38.055 INFO:tasks.rados.rados.0.smithi117.stdout:155: oid 17 updating version 34 to 35 2024-09-16T06:48:38.055 INFO:tasks.rados.rados.0.smithi117.stdout:155: oid 17 version 35 is already newer than 33 2024-09-16T06:48:38.055 INFO:tasks.rados.rados.0.smithi117.stdout:155: oid 17 version 35 is already newer than 32 2024-09-16T06:48:38.055 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 17 v 35 (ObjNum 84 snap 11 seq_num 84) dirty exists 2024-09-16T06:48:38.055 INFO:tasks.rados.rados.0.smithi117.stdout:155: left oid 17 (ObjNum 84 snap 11 seq_num 84) 2024-09-16T06:48:38.089 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:38.088+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:38.190 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:38.187+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:38.194 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.194 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.208 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:48:38.291 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:38.292+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:38.326 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.327 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:48:38.391 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:38.392+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:38.462 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.462 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.476 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:48:38.521 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:38.520+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:38.593 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:38.591+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:38.606 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.606 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.620 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:48:38.694 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:38.691+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:38.731 INFO:tasks.rados.rados.0.smithi117.stdout:145: finishing write tid 4 to smithi11731613-47 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:145: finishing write tid 5 to smithi11731613-47 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:145: oid 47 updating version 0 to 31 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:145: oid 47 updating version 31 to 33 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:145: oid 47 version 33 is already newer than 32 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:145: oid 47 version 33 is already newer than 30 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 47 v 33 (ObjNum 80 snap 11 seq_num 80) dirty exists 2024-09-16T06:48:38.734 INFO:tasks.rados.rados.0.smithi117.stdout:145: left oid 47 (ObjNum 80 snap 11 seq_num 80) 2024-09-16T06:48:38.782 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.782 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.795 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:38.791+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:38.796 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:48:38.843 INFO:tasks.rados.rados.0.smithi117.stdout:145: done (4 left) 2024-09-16T06:48:38.843 INFO:tasks.rados.rados.0.smithi117.stdout:147: done (3 left) 2024-09-16T06:48:38.844 INFO:tasks.rados.rados.0.smithi117.stdout:154: done (2 left) 2024-09-16T06:48:38.844 INFO:tasks.rados.rados.0.smithi117.stdout:155: done (1 left) 2024-09-16T06:48:38.844 INFO:tasks.rados.rados.0.smithi117.stdout:156: done (0 left) 2024-09-16T06:48:38.844 INFO:tasks.rados.rados.0.smithi117.stdout:157: snap_remove snap 3 2024-09-16T06:48:38.844 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:48:38.916 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:38.916+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:38.917 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:38.917 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:38.933 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:48:38.996 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:38.995+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:39.097 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:39.096+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:39.097 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:39.098 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:39.112 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:48:39.198 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:39.195+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:39.275 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:39.275 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:39.289 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:48:39.299 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:39.295+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:39.400 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:39.396+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:39.405 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:39.405 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:39.419 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:48:39.501 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:39.499+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:39.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:39.580 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:39.581 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:39.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:48:39.601 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:39.603+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:39.664 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:39.665 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:39.692 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:39.692 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:39.703 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:39.700+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:39.712 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:39.712 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:39.721 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:39.726 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:48:39.749 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:39.749 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:39.776 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:39.803 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:39.799+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:39.806 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:39.806 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:39.850 INFO:tasks.rados.rados.0.smithi117.stdout:157: done (0 left) 2024-09-16T06:48:39.850 INFO:tasks.rados.rados.0.smithi117.stdout:158: delete oid 46 current snap is 12 2024-09-16T06:48:39.877 INFO:tasks.rados.rados.0.smithi117.stdout:158: done (0 left) 2024-09-16T06:48:39.877 INFO:tasks.rados.rados.0.smithi117.stdout:159: snap_create 2024-09-16T06:48:39.904 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:39.903+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:39.908 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:39.908 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:39.924 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:48:40.005 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:40.003+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:40.043 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:40.043 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:40.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:48:40.106 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:40.103+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:40.178 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:40.178 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:40.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:48:40.207 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:40.203+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:40.308 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:40.307+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:40.314 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:40.314 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:40.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:48:40.409 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:40.408+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:40.460 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:40.460 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:40.461 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:40.461 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:40.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:48:40.510 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:40.507+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:40.597 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:40.597 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:40.597 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:40.597 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:40.597 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:40.611 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:48:40.612 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:40.608+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:40.629 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:40.629 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:40.629 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T06:48:40.629 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.645795 2024-09-16T06:48:40.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.6457954044511408 2024-09-16T06:48:40.639 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:40.639 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:40.669 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:40.670 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:40.696 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:40.696 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:40.712 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:40.711+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:40.730 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:40.730 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:40.739 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:40.739 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:40.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:40.739 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:40.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:70 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'44, mlcod=24'40)", 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:08.537808+0000", 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.203665110000003, 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059384349999999997, 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537808+0000", 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537808+0000", 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537807+0000", 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537816+0000", 2024-09-16T06:48:40.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1220000000000001e-06 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537868+0000", 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2414000000000003e-05 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537879+0000", 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1209999999999999e-05 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537923+0000", 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3164999999999998e-05 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537936+0000", 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3864e-05 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.543717+0000", 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057801290000000002 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.543746+0000", 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9717999999999999e-05 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:135 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'93, mlcod=25'89)", 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 31.265134477, 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073860260000000004, 2024-09-16T06:48:40.744 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476337+0000", 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476374+0000", 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7509000000000003e-05 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476453+0000", 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8273000000000001e-05 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476468+0000", 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476574+0000", 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000106879 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476624+0000", 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9910000000000002e-05 2024-09-16T06:48:40.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483640+0000", 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00701565 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483724+0000", 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4535000000000003e-05 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:141 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'99, mlcod=25'91)", 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 31.255630669999999, 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.28011762800000001, 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485841+0000", 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485852+0000", 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1313e-05 2024-09-16T06:48:40.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764267+0000", 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.278414628 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764288+0000", 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.105e-05 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764448+0000", 2024-09-16T06:48:40.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016007599999999999 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764487+0000", 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9125999999999998e-05 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765872+0000", 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001384763 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765960+0000", 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7979000000000006e-05 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:160 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'114, mlcod=26'107)", 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.811859740999999, 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061551990000000001, 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.753 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:40.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929617+0000", 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5430000000000004e-06 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929973+0000", 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035563700000000001 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929979+0000", 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7599999999999999e-06 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.931187+0000", 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001208081 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.931239+0000", 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2522999999999998e-05 2024-09-16T06:48:40.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935719+0000", 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044799219999999999 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935768+0000", 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9113000000000002e-05 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:161 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'115, mlcod=26'107)", 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.811852898000001, 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062540549999999997, 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929715+0000", 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5147000000000002e-05 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929991+0000", 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027581399999999999 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929997+0000", 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1399999999999997e-06 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932071+0000", 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020736499999999998 2024-09-16T06:48:40.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932100+0000", 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9090000000000001e-05 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935772+0000", 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0036722019999999998 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935874+0000", 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102296 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:162 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'116, mlcod=26'107)", 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.811753825, 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062203090000000003, 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:40.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929791+0000", 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1872999999999994e-05 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930051+0000", 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026016400000000001 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930059+0000", 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4100000000000008e-06 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933018+0000", 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029584709999999998 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933048+0000", 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.029e-05 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935882+0000", 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028339509999999999 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935939+0000", 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7636999999999997e-05 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:163 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'117, mlcod=26'107)", 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.811675806, 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062254820000000001, 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929801+0000", 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2030000000000002e-06 2024-09-16T06:48:40.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930091+0000", 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028969100000000002 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930097+0000", 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1870000000000002e-06 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933566+0000", 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003469663 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933600+0000", 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3293000000000001e-05 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935942+0000", 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023424190000000001 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936023+0000", 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0438999999999993e-05 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:164 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'118, mlcod=26'107)", 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.811669363, 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062774559999999998, 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929803+0000", 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929846+0000", 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3038999999999999e-05 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930109+0000", 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026273799999999999 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930115+0000", 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0900000000000001e-06 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934327+0000", 2024-09-16T06:48:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004212248 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934363+0000", 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5576e-05 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936028+0000", 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016651770000000001 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936081+0000", 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2874000000000002e-05 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:165 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'119, mlcod=26'107)", 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.811623619999999, 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0069224810000000003, 2024-09-16T06:48:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929854+0000", 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7600000000000002e-06 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930127+0000", 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000273374 2024-09-16T06:48:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930135+0000", 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.157e-06 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935309+0000", 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051733370000000001 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935340+0000", 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1423000000000001e-05 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936696+0000", 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013555730000000001 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936772+0000", 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6260000000000005e-05 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.8s2 35/32 ECSubWrite(tid=159, reqid=client.4511.0:118, at_version=35'3, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.027251+0000", 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 17.714221628000001, 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598439, 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027251+0000", 2024-09-16T06:48:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027251+0000", 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027250+0000", 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027487+0000", 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023623000000000001 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027597+0000", 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011027399999999999 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027627+0000", 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9808e-05 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027701+0000", 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4275000000000003e-05 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027710+0000", 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.5890000000000003e-06 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.033225+0000", 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005515041 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.033236+0000", 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1001e-05 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=32, reqid=client.4511.0:237, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.779347635000001, 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.008178467, 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.785 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962124+0000", 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962148+0000", 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3572999999999999e-05 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962287+0000", 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013903200000000001 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962295+0000", 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1999999999999994e-06 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962430+0000", 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135129 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962474+0000", 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3503e-05 2024-09-16T06:48:40.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970216+0000", 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077425230000000003 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7330000000000006e-05 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=34, reqid=client.4511.0:228, at_version=40'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.778598904999999, 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075127500000000003, 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962872+0000", 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962920+0000", 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7939999999999998e-05 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963011+0000", 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0655999999999994e-05 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963034+0000", 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3782999999999999e-05 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964104+0000", 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010698509999999999 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964127+0000", 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2563000000000001e-05 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061773169999999999 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970387+0000", 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2680000000000001e-05 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=35, reqid=client.4511.0:238, at_version=40'13, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.778167231999999, 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00704997, 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.794 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963304+0000", 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963342+0000", 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7753000000000003e-05 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963421+0000", 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9549999999999996e-05 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963436+0000", 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4885999999999999e-05 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964389+0000", 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095243900000000002 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964403+0000", 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.381e-05 2024-09-16T06:48:40.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970320+0000", 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059176259999999996 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970356+0000", 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5525999999999998e-05 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=36, reqid=client.4511.0:229, at_version=40'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.777658943, 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006602887, 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963812+0000", 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963860+0000", 2024-09-16T06:48:40.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7182999999999999e-05 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963954+0000", 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3912999999999993e-05 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963975+0000", 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1506e-05 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964846+0000", 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087055200000000002 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964868+0000", 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.243e-05 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970396+0000", 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005528397 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970417+0000", 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0418999999999999e-05 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=37, reqid=client.4511.0:239, at_version=40'14, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.777464991, 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070022000000000001, 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.803 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964007+0000", 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964083+0000", 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6262999999999998e-05 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964162+0000", 2024-09-16T06:48:40.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8568999999999999e-05 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964181+0000", 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9283000000000002e-05 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965031+0000", 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00084985599999999996 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965056+0000", 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4868999999999999e-05 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970969+0000", 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059132270000000001 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.971010+0000", 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1285999999999997e-05 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.807 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=77, reqid=osd.1.0:77, at_version=52'38, trim_to=0'0, roll_forward_to=50'36))", 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.87932047400000002, 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061672749999999998, 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:48:40.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862151+0000", 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862203+0000", 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2352000000000003e-05 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862291+0000", 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8163000000000003e-05 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862313+0000", 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2255999999999998e-05 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862436+0000", 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000122753 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862462+0000", 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5559999999999999e-05 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868253+0000", 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057913009999999996 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868320+0000", 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6855999999999998e-05 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=80, reqid=osd.1.0:78, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.86134201600000004, 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064748710000000001, 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.812 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880130+0000", 2024-09-16T06:48:40.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880152+0000", 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2322000000000001e-05 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880228+0000", 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6367999999999998e-05 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880245+0000", 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6186999999999999e-05 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880330+0000", 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.5568000000000005e-05 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880347+0000", 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6528000000000002e-05 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886553+0000", 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006206423 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886606+0000", 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2704999999999998e-05 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/37 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'37, trim_to=0'0, roll_forward_to=50'35))", 2024-09-16T06:48:40.816 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.85979557500000003, 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0081001449999999996, 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881676+0000", 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881715+0000", 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8512e-05 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881760+0000", 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4669999999999998e-05 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881776+0000", 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6371999999999998e-05 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881968+0000", 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019165999999999999 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881998+0000", 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0332999999999999e-05 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889735+0000", 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077367310000000002 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889778+0000", 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2766999999999999e-05 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.820 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=81, reqid=osd.1.0:81, at_version=52'42, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.858814191, 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076324610000000001, 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:48:40.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882657+0000", 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882697+0000", 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9900000000000001e-05 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882763+0000", 2024-09-16T06:48:40.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5949000000000002e-05 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882779+0000", 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6118000000000001e-05 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883074+0000", 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000294393 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883098+0000", 2024-09-16T06:48:40.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4049999999999998e-05 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890245+0000", 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071470479999999996 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890291+0000", 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6343e-05 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=82, reqid=osd.1.0:82, at_version=52'44, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.885013+0000", 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.856459737, 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060881939999999999, 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885013+0000", 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885013+0000", 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885012+0000", 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885052+0000", 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9823999999999997e-05 2024-09-16T06:48:40.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885135+0000", 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2915000000000002e-05 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885151+0000", 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6303999999999998e-05 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:40.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886686+0000", 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015355430000000001 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886710+0000", 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4063e-05 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.891058+0000", 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043475199999999997 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:40.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.891101+0000", 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3439000000000002e-05 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:40.829 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:40.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:48:40.831 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:40.831 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:40.832 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:40.832 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:40.832 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:40.833 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:40.833 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:40.828+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:40.843 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:40.843 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:40.853 INFO:tasks.rados.rados.0.smithi117.stdout:159: done (0 left) 2024-09-16T06:48:40.853 INFO:tasks.rados.rados.0.smithi117.stdout:160: copy_from oid 6 from oid 7 current snap is 13 2024-09-16T06:48:40.853 INFO:tasks.rados.rados.0.smithi117.stdout:161: delete oid 46 current snap is 13 2024-09-16T06:48:40.869 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:48:40.870 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:40.871 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:40.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:40.872+0000 7fc78ffff640 0 -- 172.21.15.100:0/3547958664 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7fc790224250 msgr2=0x7fc790226640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:40.932 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:40.927+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:40.960 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:40.960 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:40.960 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:40.960 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:40.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:48:41.033 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:41.032+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:41.089 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:41.089 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:41.089 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:41.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:48:41.106 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:41.106 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:41.175 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:41.176+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/32 ECSubWrite(tid=2, reqid=client.4511.0:19, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.308938853000001, 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0095178020000000006, 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.221 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914681+0000", 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1100000000000003e-05 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914894+0000", 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021241899999999999 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914899+0000", 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.857e-06 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916733+0000", 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018331899999999999 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916758+0000", 2024-09-16T06:48:41.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5721999999999999e-05 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924087+0000", 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073282740000000001 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924158+0000", 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.148e-05 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/32 ECSubWrite(tid=3, reqid=client.4511.0:20, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.308889021999999, 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0095058429999999999, 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914700+0000", 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0224000000000001e-05 2024-09-16T06:48:41.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914912+0000", 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021219500000000001 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914929+0000", 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6424999999999999e-05 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917780+0000", 2024-09-16T06:48:41.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851494 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917803+0000", 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3071999999999999e-05 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924166+0000", 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063629560000000003 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924196+0000", 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9638999999999999e-05 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 35/32 ECSubWrite(tid=4, reqid=client.4511.0:25, at_version=35'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.916464+0000", 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.307115580000001, 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077895539999999997, 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.230 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916464+0000", 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916464+0000", 2024-09-16T06:48:41.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916463+0000", 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916594+0000", 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000131023 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916641+0000", 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7592000000000001e-05 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916650+0000", 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0529999999999996e-06 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918332+0000", 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001681788 2024-09-16T06:48:41.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918355+0000", 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2289999999999998e-05 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924200+0000", 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005845416 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924253+0000", 2024-09-16T06:48:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3173000000000001e-05 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=9, reqid=client.4511.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.950424+0000", 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.273155044999999, 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0099245310000000003, 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950424+0000", 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950424+0000", 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950423+0000", 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950440+0000", 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7235e-05 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959533+0000", 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090923519999999997 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959542+0000", 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5980000000000003e-06 2024-09-16T06:48:41.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959593+0000", 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0599999999999997e-05 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959599+0000", 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8869999999999997e-06 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.960327+0000", 2024-09-16T06:48:41.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00072782399999999996 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.960349+0000", 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2076999999999999e-05 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=12, reqid=client.4511.0:68, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:48:41.239 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.262489063, 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.041915173, 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961089+0000", 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961108+0000", 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8260000000000001e-05 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001032+0000", 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.039924492999999998 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001043+0000", 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1063000000000001e-05 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001095+0000", 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1703999999999998e-05 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001102+0000", 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8360000000000003e-06 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002963+0000", 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018614529999999999 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.003005+0000", 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2182999999999999e-05 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.243 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=14, reqid=client.4511.0:98, at_version=35'10, trim_to=0'0, roll_forward_to=35'7))", 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.221415311000001, 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082808355, 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:48:41.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002163+0000", 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002178+0000", 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4431e-05 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084421+0000", 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082243256000000001 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084430+0000", 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8880000000000003e-06 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084484+0000", 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4193000000000001e-05 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084496+0000", 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1754e-05 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084946+0000", 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000450242 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084972+0000", 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6281000000000001e-05 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=16, reqid=client.4511.0:158, at_version=35'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.139195182000002, 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41790576000000001, 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.248 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084383+0000", 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084401+0000", 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7584999999999999e-05 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500734+0000", 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41633377199999999 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500752+0000", 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7374e-05 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500862+0000", 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010999500000000001 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500877+0000", 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5437000000000001e-05 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502236+0000", 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001358389 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502290+0000", 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4076000000000002e-05 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.cs4 35/32 ECSubWrite(tid=30, reqid=client.4511.0:160, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.514576+0000", 2024-09-16T06:48:41.252 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 17.709003506999998, 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0068807809999999999, 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.514576+0000", 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.514576+0000", 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.514566+0000", 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.99999 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.514769+0000", 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020306100000000001 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.514905+0000", 2024-09-16T06:48:41.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013589900000000001 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.514922+0000", 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7527e-05 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.515050+0000", 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012767900000000001 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.515077+0000", 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7322999999999999e-05 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.521394+0000", 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063173359999999998 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.521456+0000", 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2015999999999997e-05 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.256 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:244 3.7s0 3:e60b6e1b:::smithi11731613-14:head [stat,delete] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.982404+0000", 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.241174880000001, 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086814219999999994, 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 244 2024-09-16T06:48:41.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982404+0000", 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982404+0000", 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982392+0000", 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999876 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982419+0000", 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7166e-05 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982460+0000", 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1303000000000001e-05 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982482+0000", 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2116999999999998e-05 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982639+0000", 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015620200000000001 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982907+0000", 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026842000000000001 2024-09-16T06:48:41.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985003+0000", 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020963589999999999 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.987604+0000", 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026004349999999999 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:41.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.990914+0000", 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033104649999999998 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.991086+0000", 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017147500000000001 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:249 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~704512] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.057326+0000", 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.166253308, 2024-09-16T06:48:41.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010497131999999999, 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 249 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.057326+0000", 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.057326+0000", 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.057323+0000", 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059497+0000", 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002174175 2024-09-16T06:48:41.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059528+0000", 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0790000000000002e-05 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059544+0000", 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6439999999999998e-05 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059635+0000", 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1281999999999998e-05 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059809+0000", 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017336200000000001 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.061631+0000", 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018220210000000001 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067293+0000", 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056618830000000004 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067733+0000", 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044024500000000002 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067823+0000", 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0100999999999997e-05 2024-09-16T06:48:41.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:250 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~376832] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.062774+0000", 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.160805614999999, 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062421860000000003, 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 250 2024-09-16T06:48:41.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.062774+0000", 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.062774+0000", 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.062771+0000", 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063364+0000", 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000593455 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063390+0000", 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5406999999999999e-05 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063403+0000", 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3190000000000001e-05 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063516+0000", 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112735 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063672+0000", 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015642600000000001 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.065131+0000", 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0014585570000000001 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067439+0000", 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023082179999999999 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:41.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068922+0000", 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001483173 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.069016+0000", 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3595000000000001e-05 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:251 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (56) in=7b] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.063422+0000", 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.160156957, 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064518809999999996, 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 251 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063422+0000", 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063422+0000", 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063421+0000", 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063431+0000", 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0553e-05 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063450+0000", 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8459999999999999e-05 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063463+0000", 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.342e-05 2024-09-16T06:48:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.066434+0000", 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029705589999999998 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.066533+0000", 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9739000000000002e-05 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.066971+0000", 2024-09-16T06:48:41.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000437138 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068567+0000", 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001596011 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.069786+0000", 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012192119999999999 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.069874+0000", 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8314999999999997e-05 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:252 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.160062431, 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.38343603199999998, 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.279 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 252 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063515+0000", 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063523+0000", 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7430000000000002e-06 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063545+0000", 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1310000000000001e-05 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063562+0000", 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7629999999999999e-05 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068202+0000", 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046400269999999997 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068467+0000", 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000264691 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070315+0000", 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018477190000000001 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070443+0000", 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012795699999999999 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.446953+0000", 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.376510228 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:273 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~819200,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:29.968535+0000", 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.255043911, 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191088, 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 273 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968535+0000", 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968535+0000", 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968532+0000", 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:41.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968550+0000", 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8558000000000001e-05 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968579+0000", 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8727000000000001e-05 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968591+0000", 2024-09-16T06:48:41.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2113000000000001e-05 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968683+0000", 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1625000000000005e-05 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op must wait for map", 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968697+0000", 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4331999999999999e-05 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971175+0000", 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024773120000000002 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971271+0000", 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6089999999999996e-05 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.975726+0000", 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044557590000000001 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.289 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:274 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~819200,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.254973161000001, 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0080167260000000001, 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 274 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.290 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968610+0000", 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.51e-06 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968615+0000", 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3759999999999999e-06 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968619+0000", 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.861e-06 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968815+0000", 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019582399999999999 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968824+0000", 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3470000000000005e-06 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971697+0000", 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028724000000000002 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971791+0000", 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4116000000000005e-05 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.976623+0000", 2024-09-16T06:48:41.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048317769999999998 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:275 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~819200,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:29.968674+0000", 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.254905395, 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0088331550000000005, 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.295 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 275 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968674+0000", 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968674+0000", 2024-09-16T06:48:41.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968673+0000", 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968677+0000", 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1590000000000002e-06 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968684+0000", 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9310000000000008e-06 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968689+0000", 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1640000000000003e-06 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968863+0000", 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017429899999999999 2024-09-16T06:48:41.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968872+0000", 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7879999999999995e-06 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.972130+0000", 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032578910000000002 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.972206+0000", 2024-09-16T06:48:41.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6421000000000006e-05 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.977507+0000", 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053008990000000004 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 52/37 ECSubWrite(tid=113, reqid=osd.11.0:113, at_version=52'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.861506+0000", 2024-09-16T06:48:41.300 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.362073504, 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071674260000000002, 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861506+0000", 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861506+0000", 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861503+0000", 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861557+0000", 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3959999999999998e-05 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861663+0000", 2024-09-16T06:48:41.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105312 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861685+0000", 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1877000000000001e-05 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861883+0000", 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019818099999999999 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861898+0000", 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4895999999999999e-05 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867872+0000", 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059742629999999996 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868673+0000", 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000801244 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.304 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.cs4 52/37 ECSubWrite(tid=116, reqid=osd.11.0:116, at_version=52'39, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.865489+0000", 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.3580899550000001, 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070676209999999996, 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865489+0000", 2024-09-16T06:48:41.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865489+0000", 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865488+0000", 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865537+0000", 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9320000000000002e-05 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865628+0000", 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1471999999999994e-05 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865650+0000", 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1650000000000001e-05 2024-09-16T06:48:41.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865846+0000", 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019564499999999999 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.865870+0000", 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4513000000000001e-05 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.872495+0000", 2024-09-16T06:48:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0066245979999999998 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.872557+0000", 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2013000000000004e-05 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.2s3 52/32 ECSubWrite(tid=64, reqid=osd.14.0:64, at_version=52'52, trim_to=0'0, roll_forward_to=50'48))", 2024-09-16T06:48:41.309 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.867665+0000", 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.355914176, 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006669716, 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867665+0000", 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867665+0000", 2024-09-16T06:48:41.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867663+0000", 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867714+0000", 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0722e-05 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867814+0000", 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010055 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867870+0000", 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5597999999999999e-05 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868087+0000", 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021726199999999999 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868113+0000", 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6230000000000001e-05 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.874291+0000", 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061778720000000001 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.874335+0000", 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3402000000000002e-05 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.313 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 52/37 ECSubWrite(tid=117, reqid=osd.11.0:115, at_version=0'0, trim_to=0'0, roll_forward_to=52'54))", 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.879370+0000", 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.3442087250000001, 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0085184830000000003, 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879370+0000", 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879370+0000", 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879369+0000", 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879389+0000", 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9582e-05 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879463+0000", 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3619000000000001e-05 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879479+0000", 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6543999999999999e-05 2024-09-16T06:48:41.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879799+0000", 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032016100000000001 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879825+0000", 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.527e-05 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882406+0000", 2024-09-16T06:48:41.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002581131 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.887889+0000", 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054831960000000001 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.318 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:41.320 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:48:41.320 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:41.319+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:41.420 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:41.416+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:41.479 INFO:tasks.rados.rados.0.smithi117.stdout:161: done (1 left) 2024-09-16T06:48:41.480 INFO:tasks.rados.rados.0.smithi117.stdout:162: snap_remove snap 12 2024-09-16T06:48:41.491 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:41.491 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:41.491 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:41.492 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:41.506 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:48:41.521 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:41.519+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:41.622 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:41.623+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:41.640 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:48:41.723 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:41.723+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.bs4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:16, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.767 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.904207+0000", 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.863727681, 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042589339999999998, 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904207+0000", 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904207+0000", 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904206+0000", 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904423+0000", 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021709499999999999 2024-09-16T06:48:41.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904494+0000", 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0798999999999998e-05 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904533+0000", 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8930000000000002e-05 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904695+0000", 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000161755 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.904710+0000", 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5197e-05 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.908370+0000", 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0036603989999999999 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.771 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.908466+0000", 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5421999999999995e-05 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.bs4 34/32 ECSubWrite(tid=3, reqid=client.4511.0:17, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.906880+0000", 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.861054313, 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003481006, 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.772 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.906880+0000", 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.906880+0000", 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.906878+0000", 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.906926+0000", 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7960000000000002e-05 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.907032+0000", 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000106229 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.907059+0000", 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.739e-05 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.907194+0000", 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013496499999999999 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.907214+0000", 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9677000000000002e-05 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.910307+0000", 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030926650000000001 2024-09-16T06:48:41.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.910361+0000", 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4206999999999998e-05 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=4, reqid=client.4511.0:40, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.777 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.849765403999999, 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055223370000000004, 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:41.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918371+0000", 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020196999999999999 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918465+0000", 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4196000000000004e-05 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918494+0000", 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9254000000000001e-05 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918552+0000", 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8031000000000001e-05 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918561+0000", 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3950000000000007e-06 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.923659+0000", 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050976270000000004 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.923691+0000", 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2160999999999999e-05 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.781 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=5, reqid=client.4511.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918376+0000", 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.849558201000001, 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035267229999999998, 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.783 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918376+0000", 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918376+0000", 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918375+0000", 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918566+0000", 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019061600000000001 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918607+0000", 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1180000000000002e-05 2024-09-16T06:48:41.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918616+0000", 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7469999999999994e-06 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918744+0000", 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012794600000000001 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918768+0000", 2024-09-16T06:48:41.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3896999999999998e-05 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921859+0000", 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030907270000000001 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921903+0000", 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4302999999999999e-05 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.787 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.as4 35/32 ECSubWrite(tid=4, reqid=client.4511.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.920640+0000", 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.847294459, 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031369150000000001, 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920640+0000", 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920640+0000", 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920639+0000", 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920657+0000", 2024-09-16T06:48:41.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8686999999999999e-05 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920711+0000", 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3226999999999997e-05 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920732+0000", 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1109000000000001e-05 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920807+0000", 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5618999999999995e-05 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920818+0000", 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.923712+0000", 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028939730000000002 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.923777+0000", 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4746000000000007e-05 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=6, reqid=client.4511.0:41, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.921808+0000", 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.846125815000001, 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030595219999999998, 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.792 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921808+0000", 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921808+0000", 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921808+0000", 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921894+0000", 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6769999999999998e-05 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921994+0000", 2024-09-16T06:48:41.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9940999999999997e-05 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922005+0000", 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0847e-05 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922062+0000", 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6882000000000002e-05 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922072+0000", 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5999999999999996e-06 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924827+0000", 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027549739999999999 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924868+0000", 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1489999999999997e-05 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.bs4 35/32 ECSubWrite(tid=7, reqid=client.4511.0:38, at_version=35'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.921900+0000", 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.846034110000002, 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028984990000000001, 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921900+0000", 2024-09-16T06:48:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921900+0000", 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921900+0000", 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.921933+0000", 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2929999999999998e-05 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922024+0000", 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1435999999999997e-05 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922030+0000", 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1310000000000003e-06 2024-09-16T06:48:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922081+0000", 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1147999999999999e-05 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.922089+0000", 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4789999999999999e-06 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924776+0000", 2024-09-16T06:48:41.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026872630000000001 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924799+0000", 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2722e-05 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=11, reqid=client.4511.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.935919+0000", 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.832015701, 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050956999999999999, 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935919+0000", 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935919+0000", 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935917+0000", 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935936+0000", 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8315e-05 2024-09-16T06:48:41.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935988+0000", 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2194000000000002e-05 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936000+0000", 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2277e-05 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936118+0000", 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117835 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936141+0000", 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3425999999999999e-05 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.940932+0000", 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004790356 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941014+0000", 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2599e-05 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:52 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.831825555999998, 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011921681999999999, 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:41.806 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 52 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936107+0000", 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936869+0000", 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076224400000000003 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936883+0000", 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3704e-05 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936893+0000", 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0567000000000001e-05 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937221+0000", 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032796600000000001 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937632+0000", 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00041103599999999999 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939883+0000", 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022507009999999999 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943370+0000", 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034869190000000002 2024-09-16T06:48:41.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.947880+0000", 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045105839999999998 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.948030+0000", 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015010800000000001 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:53 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.831024282000001, 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.679887028, 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 53 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936909+0000", 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936915+0000", 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8130000000000001e-06 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936921+0000", 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9939999999999997e-06 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936925+0000", 2024-09-16T06:48:41.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9920000000000001e-06 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941426+0000", 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045008390000000004 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941657+0000", 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023140500000000001 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.942737+0000", 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001079637 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943730+0000", 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099337199999999996 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616662+0000", 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67293178399999998 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616797+0000", 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135186 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:54 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.831006047999999, 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2642334100000001, 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 54 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936930+0000", 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.725e-06 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936933+0000", 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.698e-06 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936967+0000", 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3618000000000002e-05 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943520+0000", 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065532869999999997 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943693+0000", 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017226899999999999 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617527+0000", 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67383411699999995 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617765+0000", 2024-09-16T06:48:41.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023803499999999999 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201162+0000", 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58339697800000001 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=20, reqid=client.4511.0:110, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.017822+0000", 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.750112732000002, 2024-09-16T06:48:41.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042424129999999996, 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.017822+0000", 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.017822+0000", 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.017821+0000", 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.017833+0000", 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1769e-05 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.017855+0000", 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1773e-05 2024-09-16T06:48:41.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.017859+0000", 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0679999999999998e-06 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.018279+0000", 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042018900000000001 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.018308+0000", 2024-09-16T06:48:41.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9323e-05 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.022010+0000", 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0037019380000000001 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.022064+0000", 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3900000000000002e-05 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=32, reqid=client.4511.0:148, at_version=35'7, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.074540+0000", 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.693394153, 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029646130000000001, 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074540+0000", 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074540+0000", 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074539+0000", 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074555+0000", 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5466999999999999e-05 2024-09-16T06:48:41.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074580+0000", 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4865999999999999e-05 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074585+0000", 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2889999999999997e-06 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074628+0000", 2024-09-16T06:48:41.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2760999999999999e-05 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.074637+0000", 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6970000000000005e-06 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077479+0000", 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028415760000000002 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077505+0000", 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5879000000000001e-05 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.es0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 18.148956193, 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58220860799999996, 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.831 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618977+0000", 2024-09-16T06:48:41.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618996+0000", 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9266000000000002e-05 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200456+0000", 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58146017500000002 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200480+0000", 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3186000000000001e-05 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200604+0000", 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012390599999999999 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201187+0000", 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00058315500000000002 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.834 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 40/37 ECSubWrite(tid=20, reqid=client.4511.0:219, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.787062710000001, 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005811274, 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:48:41.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980871+0000", 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981331+0000", 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046021300000000001 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981469+0000", 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138088 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981496+0000", 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7353000000000001e-05 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981653+0000", 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000156816 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981681+0000", 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.73e-05 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986615+0000", 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049340180000000001 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986683+0000", 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8258999999999998e-05 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 40/37 ECSubWrite(tid=5, reqid=client.4511.0:244, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.985009+0000", 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.782925596, 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00359853, 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.839 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985009+0000", 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985009+0000", 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985008+0000", 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985064+0000", 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6610000000000002e-05 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985181+0000", 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116673 2024-09-16T06:48:41.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985204+0000", 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3073000000000001e-05 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985491+0000", 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028702699999999997 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985508+0000", 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7163000000000002e-05 2024-09-16T06:48:41.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.988535+0000", 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003027138 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.988607+0000", 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1996000000000007e-05 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 42/37 ECSubWrite(tid=22, reqid=client.4511.0:316, at_version=42'14, trim_to=0'0, roll_forward_to=40'12))", 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:30.176990+0000", 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.590944469, 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027976749999999999, 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.176990+0000", 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.176990+0000", 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.176989+0000", 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.177231+0000", 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024138799999999999 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.177253+0000", 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2745e-05 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.177267+0000", 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3079e-05 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.177369+0000", 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010266400000000001 2024-09-16T06:48:41.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.177395+0000", 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5795999999999998e-05 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.179755+0000", 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023599440000000001 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.179788+0000", 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2570000000000002e-05 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.as4 47/32 ECSubWrite(tid=65, reqid=client.4511.0:359, at_version=47'45, trim_to=0'0, roll_forward_to=44'43))", 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:34.861973+0000", 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9059608209999999, 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002875145, 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.861973+0000", 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.861973+0000", 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.861973+0000", 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:41.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.862035+0000", 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1934999999999996e-05 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.862135+0000", 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9872999999999997e-05 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.862152+0000", 2024-09-16T06:48:41.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6892999999999999e-05 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.862248+0000", 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6736000000000004e-05 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.862264+0000", 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5763000000000001e-05 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.864796+0000", 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025322790000000001 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.864849+0000", 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2296000000000002e-05 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.as4 47/32 ECSubWrite(tid=66, reqid=client.4511.0:360, at_version=47'46, trim_to=0'0, roll_forward_to=44'43))", 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:34.865392+0000", 2024-09-16T06:48:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9025419069999998, 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029524210000000002, 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865392+0000", 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865392+0000", 2024-09-16T06:48:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865391+0000", 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865428+0000", 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6671999999999997e-05 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865488+0000", 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9953e-05 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865502+0000", 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4194e-05 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865622+0000", 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000120405 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.865637+0000", 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4927e-05 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.868287+0000", 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026497999999999999 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.868345+0000", 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7763000000000002e-05 2024-09-16T06:48:41.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 49/37 ECSubWrite(tid=34, reqid=osd.7.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=49'22))", 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:36.861594+0000", 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.906340481, 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050575910000000002, 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861594+0000", 2024-09-16T06:48:41.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861594+0000", 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861593+0000", 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861602+0000", 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0359999999999995e-06 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861654+0000", 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2295e-05 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861662+0000", 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6000000000000001e-06 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861710+0000", 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7537000000000002e-05 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861719+0000", 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5070000000000006e-06 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.866627+0000", 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049081480000000002 2024-09-16T06:48:41.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.866651+0000", 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4226000000000001e-05 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:41.861 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:41.862 INFO:tasks.rados.rados.0.smithi117.stdout:162: done (1 left) 2024-09-16T06:48:41.863 INFO:tasks.rados.rados.0.smithi117.stdout:163: snap_create 2024-09-16T06:48:41.863 INFO:tasks.rados.rados.0.smithi117.stdout:160: finishing copy_from racing read to smithi11731613-6 2024-09-16T06:48:41.863 INFO:tasks.rados.rados.0.smithi117.stdout:160: finishing copy_from to smithi11731613-6 2024-09-16T06:48:41.863 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 6 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:41.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:48:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.7 primary-affinity to 0.645795 (8423222) 2024-09-16T06:48:41.865 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:48:41.866 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:41.859+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:41.964 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:41.963+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:42.034 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:42.034 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:42.035 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:42.035 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:42.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:48:42.097 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:42.096+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:42.198 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:42.198 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:42.198 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:42.198 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:42.198 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:42.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:48:42.199 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:42.195+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:42.298 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:42.296+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:42.367 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:42.367 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:42.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:46 3.0s0 3:043b7e8c:::smithi11731613-16:head [append 0~262144,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.917095+0000", 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.450984849000001, 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014697692, 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 46 2024-09-16T06:48:42.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917095+0000", 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917095+0000", 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917094+0000", 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918052+0000", 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095875799999999998 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918068+0000", 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5522000000000001e-05 2024-09-16T06:48:42.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918077+0000", 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918141+0000", 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4046000000000003e-05 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918300+0000", 2024-09-16T06:48:42.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159481 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.919368+0000", 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001067337 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924493+0000", 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051250860000000001 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.931676+0000", 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071828500000000002 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.931793+0000", 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117336 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.373 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:47 3.0s0 3:043b7e8c:::smithi11731613-16:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918091+0000", 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.449989626000001, 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.01424603, 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 47 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918091+0000", 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918091+0000", 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918090+0000", 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918096+0000", 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3450000000000002e-06 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918105+0000", 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.149e-06 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918111+0000", 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1299999999999998e-06 2024-09-16T06:48:42.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.919700+0000", 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001588423 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.919759+0000", 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8881000000000001e-05 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920270+0000", 2024-09-16T06:48:42.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051165300000000004 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924494+0000", 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042238670000000001 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.932107+0000", 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076127310000000002 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.932337+0000", 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000229788 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:48 3.0s0 3:043b7e8c:::smithi11731613-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.449225392999999, 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.016720260000000001, 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.379 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 48 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918854+0000", 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918860+0000", 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3960000000000003e-06 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918873+0000", 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2485e-05 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918923+0000", 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0386999999999997e-05 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920454+0000", 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015303949999999999 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920536+0000", 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2573999999999999e-05 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.933186+0000", 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.012650051000000001 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.933335+0000", 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000148689 2024-09-16T06:48:42.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935575+0000", 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022402540000000001 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:58 3.0s0 3:0900ee20:::smithi11731613-20:head [append 0~327680,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939020+0000", 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.429060643, 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010074863999999999, 2024-09-16T06:48:42.384 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 58 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939020+0000", 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939020+0000", 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939018+0000", 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939352+0000", 2024-09-16T06:48:42.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000333942 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939365+0000", 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2680000000000001e-05 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939371+0000", 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7780000000000004e-06 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939468+0000", 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7725000000000004e-05 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.940250+0000", 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078188699999999999 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.942101+0000", 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001850787 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944793+0000", 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002692177 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.949042+0000", 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042484230000000003 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.949094+0000", 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2833000000000001e-05 2024-09-16T06:48:42.389 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:59 3.0s0 3:0900ee20:::smithi11731613-20:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.428700423999999, 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.069099573999999997, 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.390 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 59 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939379+0000", 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939383+0000", 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9040000000000002e-06 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939390+0000", 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:48:42.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939394+0000", 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9380000000000002e-06 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943863+0000", 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044688669999999996 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943950+0000", 2024-09-16T06:48:42.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7173e-05 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944394+0000", 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044454100000000001 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944887+0000", 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000492156 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008423+0000", 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.063536255 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008479+0000", 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6462999999999998e-05 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:60 3.0s0 3:0900ee20:::smithi11731613-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:48:42.395 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.428638464999999, 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.076005592999999996, 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 60 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939441+0000", 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939445+0000", 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7340000000000002e-06 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939451+0000", 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.241e-06 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939457+0000", 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4580000000000001e-06 2024-09-16T06:48:42.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944649+0000", 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051924110000000001 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944778+0000", 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012887900000000001 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008904+0000", 2024-09-16T06:48:42.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.064125946000000003 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.009068+0000", 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016377000000000001 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.015447+0000", 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063797940000000003 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:100 3.9s0 3:92c79891:::smithi11731613-34:head [append 0~557056,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.001709+0000", 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.366371378, 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094224180000000001, 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 100 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.401 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001709+0000", 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001709+0000", 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001707+0000", 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002242+0000", 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053474100000000004 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002260+0000", 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8059000000000001e-05 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002269+0000", 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8920000000000006e-06 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002388+0000", 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118575 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002742+0000", 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000354304 2024-09-16T06:48:42.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.004837+0000", 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002094912 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007620+0000", 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027829949999999999 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.011096+0000", 2024-09-16T06:48:42.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034766269999999999 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.011131+0000", 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4795000000000001e-05 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:101 3.9s0 3:92c79891:::smithi11731613-34:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.365797782000001, 2024-09-16T06:48:42.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011878148999999999, 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 101 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002286+0000", 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4780000000000002e-06 2024-09-16T06:48:42.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002293+0000", 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0879999999999997e-06 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002299+0000", 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5509999999999999e-06 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.006252+0000", 2024-09-16T06:48:42.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003953397 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.006430+0000", 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017774800000000001 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007019+0000", 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00058882300000000004 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.009099+0000", 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020800160000000001 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014089+0000", 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049907900000000002 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014161+0000", 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1157999999999999e-05 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.411 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:102 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.365715252000001, 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.046755945, 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.412 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 102 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002364+0000", 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002370+0000", 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5459999999999999e-06 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002385+0000", 2024-09-16T06:48:42.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5724000000000001e-05 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002394+0000", 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3280000000000006e-06 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007411+0000", 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050172059999999997 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007528+0000", 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117267 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014384+0000", 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006855809 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014457+0000", 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2799999999999994e-05 2024-09-16T06:48:42.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049121+0000", 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.034664014 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:127 3.0s0 3:03b32ae3:::smithi11731613-43:head [append 0~720896,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.039739+0000", 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.328340809, 2024-09-16T06:48:42.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011181432999999999, 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 127 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.039739+0000", 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.039739+0000", 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.039738+0000", 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046439+0000", 2024-09-16T06:48:42.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006700981 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046452+0000", 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2411e-05 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046459+0000", 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7859999999999997e-06 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046623+0000", 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012280700000000001 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.047810+0000", 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011871679999999999 2024-09-16T06:48:42.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050744+0000", 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002934 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050881+0000", 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013747999999999999 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050921+0000", 2024-09-16T06:48:42.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9391999999999998e-05 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:128 3.0s0 3:03b32ae3:::smithi11731613-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.321607656000001, 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.031368511000000002, 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.423 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 128 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046472+0000", 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046476+0000", 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.578e-06 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046485+0000", 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2289999999999992e-06 2024-09-16T06:48:42.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046491+0000", 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6319999999999996e-06 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048421+0000", 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019297240000000001 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048515+0000", 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4073999999999994e-05 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048890+0000", 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000375306 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050760+0000", 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018703350000000001 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077776+0000", 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.027015092000000001 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077841+0000", 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5504000000000002e-05 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.428 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:129 3.0s0 3:03b32ae3:::smithi11731613-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.321580636, 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.036392558999999998, 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 129 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046499+0000", 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046502+0000", 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8200000000000001e-06 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046507+0000", 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3510000000000001e-06 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046512+0000", 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.464e-06 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049021+0000", 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025094789999999998 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049099+0000", 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7998999999999996e-05 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077886+0000", 2024-09-16T06:48:42.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.028786875 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077945+0000", 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8941999999999999e-05 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.082892+0000", 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049470909999999998 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.433 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:157 3.0s0 3:043b7e8c:::smithi11731613-16:head [read 0~262144,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.284931233000002, 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14327616900000001, 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 157 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083148+0000", 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083152+0000", 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1920000000000001e-06 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083158+0000", 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1959999999999996e-06 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083163+0000", 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2909999999999998e-06 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084618+0000", 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0014551430000000001 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084738+0000", 2024-09-16T06:48:42.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119295 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.226425+0000", 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14168755499999999 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s1 40/37 ECSubWrite(tid=40, reqid=client.4511.0:240, at_version=40'16, trim_to=0'0, roll_forward_to=40'14))", 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.978720+0000", 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 14.389360083, 2024-09-16T06:48:42.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0083299289999999998, 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.978720+0000", 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.978720+0000", 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.978719+0000", 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.979434+0000", 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071520799999999995 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.979549+0000", 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115145 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.979576+0000", 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6976999999999999e-05 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.979696+0000", 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012006900000000001 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.979725+0000", 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8543000000000001e-05 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986988+0000", 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072632479999999999 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.987050+0000", 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1952999999999995e-05 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.442 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds3 40/37 ECSubWrite(tid=20, reqid=client.4511.0:219, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.980904+0000", 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 14.387175952, 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077543040000000001, 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980904+0000", 2024-09-16T06:48:42.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980904+0000", 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980902+0000", 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981251+0000", 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00034828099999999998 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981368+0000", 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116975 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981390+0000", 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2013000000000001e-05 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981527+0000", 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000137856 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981552+0000", 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4153000000000001e-05 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.988596+0000", 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070445630000000002 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.988659+0000", 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2342999999999996e-05 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:291 3.0s0 3:03b32ae3:::smithi11731613-43:head [rollback 7] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:30.104130+0000", 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.263950246, 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0079210510000000001, 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.447 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 291 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104130+0000", 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104130+0000", 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104129+0000", 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104133+0000", 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9570000000000004e-06 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104144+0000", 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0774000000000001e-05 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104150+0000", 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1369999999999999e-06 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.106128+0000", 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001977674 2024-09-16T06:48:42.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.106258+0000", 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013023099999999999 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.106976+0000", 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000717456 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.109463+0000", 2024-09-16T06:48:42.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024872060000000001 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.112013+0000", 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025504 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.112051+0000", 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7833999999999997e-05 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:292 3.0s0 3:03b32ae3:::smithi11731613-43:head [append 0~0] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.263922914, 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011795597, 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 292 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104160+0000", 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5820000000000002e-06 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104169+0000", 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4260000000000003e-06 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104175+0000", 2024-09-16T06:48:42.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6710000000000002e-06 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.107609+0000", 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003433204 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.108605+0000", 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099621299999999992 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.109122+0000", 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051705200000000005 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.114736+0000", 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005614165 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.115910+0000", 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011743229999999999 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.115953+0000", 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2521000000000002e-05 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:358 3.0s0 3:03b32ae3:::smithi11731613-43:head [delete] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:34.845297+0000", 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 7.5227827109999996, 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014407524, 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.458 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 358 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845297+0000", 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845297+0000", 2024-09-16T06:48:42.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845294+0000", 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845313+0000", 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9170000000000001e-05 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845350+0000", 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6766e-05 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845368+0000", 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.772e-05 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845475+0000", 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010686199999999999 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op must wait for map", 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845491+0000", 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.694e-05 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.851263+0000", 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057716750000000004 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.851521+0000", 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025770699999999998 2024-09-16T06:48:42.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.853488+0000", 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001967594 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.856388+0000", 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002899502 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.859532+0000", 2024-09-16T06:48:42.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031442229999999998 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.859705+0000", 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017284200000000001 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 49/32 ECSubWriteReply(tid=37, last_complete=49'17, committed=1, applied=1))", 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:36.846864+0000", 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.5212160109999999, 2024-09-16T06:48:42.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0084584770000000007, 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.846864+0000", 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.846864+0000", 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.846863+0000", 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.846868+0000", 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2140000000000004e-06 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855255+0000", 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0083864649999999992 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855263+0000", 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6750000000000002e-06 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855296+0000", 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3540000000000001e-05 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855323+0000", 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6503999999999999e-05 2024-09-16T06:48:42.467 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.cs3 49/37 ECSubWrite(tid=98, reqid=osd.11.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=49'26))", 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.5128434750000004, 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14832383499999999, 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855241+0000", 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5889999999999996e-06 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002739+0000", 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14749786200000001 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002747+0000", 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5390000000000002e-06 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002811+0000", 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4498999999999994e-05 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002824+0000", 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2883e-05 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.003527+0000", 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070348000000000003 2024-09-16T06:48:42.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.003561+0000", 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3121999999999999e-05 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:42.472 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:42.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:48:42.474 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:42.471+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:42.575 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:42.576+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:42.606 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:42.606 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:42.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:48:42.676 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:42.675+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:42.740 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:42.740 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:42.740 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:42.740 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:42.740 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:42.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:48:42.776 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:42.775+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:42.813 INFO:tasks.rados.rados.0.smithi117.stdout:160: done (1 left) 2024-09-16T06:48:42.813 INFO:tasks.rados.rados.0.smithi117.stdout:163: done (0 left) 2024-09-16T06:48:42.813 INFO:tasks.rados.rados.0.smithi117.stdout:164: delete oid 9 current snap is 14 2024-09-16T06:48:42.878 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:42.875+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:10 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.985422700000001, 2024-09-16T06:48:42.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025558260999999999, 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896131+0000", 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896384+0000", 2024-09-16T06:48:42.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025249400000000003 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896423+0000", 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9551999999999997e-05 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896446+0000", 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2657999999999999e-05 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896601+0000", 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015511400000000001 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896938+0000", 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000336779 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899233+0000", 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022945840000000001 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903156+0000", 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039230970000000004 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921526+0000", 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018370496 2024-09-16T06:48:42.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921693+0000", 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000166916 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:11 3.4s0 3:2be2ffb2:::smithi11731613-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.985070096000001, 2024-09-16T06:48:42.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025954171000000002, 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896485+0000", 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896497+0000", 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1669e-05 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896518+0000", 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1180999999999999e-05 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896534+0000", 2024-09-16T06:48:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5732000000000001e-05 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900249+0000", 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037148250000000002 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900475+0000", 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022637300000000001 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901657+0000", 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001182249 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.909750+0000", 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0080923149999999992 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922220+0000", 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012470623 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922442+0000", 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022146399999999999 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:12 3.4s0 3:2be2ffb2:::smithi11731613-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.984968735999999, 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032662215000000001, 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 12 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896587+0000", 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896595+0000", 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9389999999999997e-06 2024-09-16T06:48:42.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896616+0000", 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0571e-05 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896631+0000", 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5315000000000002e-05 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057790200000000002 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902594+0000", 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018354499999999999 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922598+0000", 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020004152000000001 2024-09-16T06:48:42.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923951+0000", 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001352771 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929251+0000", 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005300389 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:34 3.4s0 3:2123e90c:::smithi11731613-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.980466455999998, 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031689311999999997, 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 34 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901089+0000", 2024-09-16T06:48:42.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901956+0000", 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086772399999999995 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901994+0000", 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7880000000000003e-05 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902023+0000", 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8479999999999998e-05 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902695+0000", 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067224400000000001 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902713+0000", 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7719000000000001e-05 2024-09-16T06:48:42.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.910786+0000", 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00807276 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911114+0000", 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032852700000000001 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914294+0000", 2024-09-16T06:48:42.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003180113 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918604+0000", 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043102669999999996 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932586+0000", 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013981214 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932780+0000", 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019474 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:35 3.4s0 3:2123e90c:::smithi11731613-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.979511736999999, 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031230615999999999, 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.907 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 35 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902044+0000", 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902055+0000", 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0961999999999999e-05 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902075+0000", 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9748000000000001e-05 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902088+0000", 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3791999999999999e-05 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001041583 2024-09-16T06:48:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903146+0000", 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6511000000000002e-05 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.915912+0000", 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012765472999999999 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916161+0000", 2024-09-16T06:48:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024903700000000002 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917305+0000", 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001143813 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927287+0000", 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099818019999999997 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933082+0000", 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057955740000000004 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933276+0000", 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019429399999999999 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:36 3.4s0 3:2123e90c:::smithi11731613-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.979089677000001, 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038105068999999998, 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 36 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902465+0000", 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902479+0000", 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3405e-05 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902529+0000", 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9638999999999997e-05 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902555+0000", 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-16T06:48:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903331+0000", 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077605999999999997 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903346+0000", 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5362999999999998e-05 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918677+0000", 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015331205000000001 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919160+0000", 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00048271000000000002 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936255+0000", 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017094597999999999 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936424+0000", 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168789 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940573+0000", 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041493600000000004 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:93 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.990651+0000", 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.890906320999999, 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011000630000000001, 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 93 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990651+0000", 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990651+0000", 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990650+0000", 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990657+0000", 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2740000000000002e-06 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990676+0000", 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8624e-05 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.990689+0000", 2024-09-16T06:48:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3141e-05 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.996905+0000", 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062163640000000003 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.997144+0000", 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238765 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.998942+0000", 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001798062 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.999158+0000", 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021595100000000001 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.001652+0000", 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002493706 2024-09-16T06:48:42.924 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:135 3.4s0 3:2cfe9c23:::smithi11731613-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.050267+0000", 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.831290851999999, 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010744566000000001, 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 135 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050267+0000", 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050267+0000", 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050266+0000", 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050272+0000", 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6989999999999997e-06 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050291+0000", 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8847000000000001e-05 2024-09-16T06:48:42.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050310+0000", 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8442999999999999e-05 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054567+0000", 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042575620000000003 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054688+0000", 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120741 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.059268+0000", 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045798139999999998 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.059338+0000", 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0190000000000004e-05 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061011+0000", 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001673384 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:227 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958230+0000", 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.923327133000001, 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0115925, 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.930 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 227 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958230+0000", 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958230+0000", 2024-09-16T06:48:42.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958227+0000", 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958247+0000", 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0268999999999999e-05 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958285+0000", 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7698999999999999e-05 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958309+0000", 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3538000000000001e-05 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958489+0000", 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017992799999999999 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958510+0000", 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1220999999999999e-05 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960761+0000", 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022513429999999998 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960816+0000", 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4796999999999997e-05 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961347+0000", 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00053147800000000005 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964120+0000", 2024-09-16T06:48:42.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027730979999999999 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.969759+0000", 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056390040000000004 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.969823+0000", 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3411999999999995e-05 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:228 3.4s0 3:2be2ffb2:::smithi11731613-4:head [rollback 5] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.923205161, 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015696132000000002, 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 228 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958355+0000", 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3280000000000002e-06 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958364+0000", 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6410000000000008e-06 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958370+0000", 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0009999999999998e-06 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958705+0000", 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033533 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958720+0000", 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4976e-05 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961715+0000", 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029947229999999999 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961796+0000", 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1070999999999997e-05 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962213+0000", 2024-09-16T06:48:42.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00041693299999999997 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964999+0000", 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027864159999999999 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974015+0000", 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090157589999999999 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974049+0000", 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3501999999999999e-05 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:229 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.923063453999999, 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015649126999999999, 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.943 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 229 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:42.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958497+0000", 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4709999999999999e-06 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958504+0000", 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1500000000000002e-06 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958510+0000", 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9399999999999999e-06 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958785+0000", 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027471700000000001 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958790+0000", 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9470000000000002e-06 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962470+0000", 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036807070000000001 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962542+0000", 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1168999999999996e-05 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962912+0000", 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037075200000000002 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965035+0000", 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021227419999999999 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974112+0000", 2024-09-16T06:48:42.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090767599999999997 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974143+0000", 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1315000000000001e-05 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:237 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:42.949 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.923038548999999, 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013107461000000001, 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 237 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958521+0000", 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6469999999999998e-06 2024-09-16T06:48:42.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958529+0000", 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8830000000000006e-06 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958535+0000", 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4410000000000001e-06 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958587+0000", 2024-09-16T06:48:42.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2215000000000001e-05 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958594+0000", 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8589999999999999e-06 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960575+0000", 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019818380000000001 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960703+0000", 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012750899999999999 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961363+0000", 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00066002300000000003 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964110+0000", 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027467659999999999 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.971588+0000", 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074782549999999996 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.971626+0000", 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8476999999999998e-05 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:238 3.1s0 3:825f7e25:::smithi11731613-3:head [rollback 2] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.923013234999999, 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014029764, 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 238 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958547+0000", 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2629999999999999e-06 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958554+0000", 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0480000000000001e-06 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958561+0000", 2024-09-16T06:48:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2919999999999998e-06 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958670+0000", 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010880800000000001 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958682+0000", 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1966e-05 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961922+0000", 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032400459999999999 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961990+0000", 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7710000000000001e-05 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962713+0000", 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000722738 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965012+0000", 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022997120000000002 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972544+0000", 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007531055 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972574+0000", 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0550999999999999e-05 2024-09-16T06:48:42.961 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:239 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.922991228000001, 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014095699999999999, 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 239 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958570+0000", 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7950000000000001e-06 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958576+0000", 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-16T06:48:42.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958583+0000", 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2629999999999997e-06 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958766+0000", 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018306299999999999 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958783+0000", 2024-09-16T06:48:42.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6801000000000001e-05 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963004+0000", 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004221168 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963078+0000", 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3843000000000004e-05 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963329+0000", 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025082499999999998 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965054+0000", 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017253220000000001 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972634+0000", 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075798469999999998 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972662+0000", 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8232999999999999e-05 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:240 3.4s0 3:2ae27109:::smithi11731613-31:head [append 0~655360] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.975800+0000", 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.905757683000001, 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055275000000001, 2024-09-16T06:48:42.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 240 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.975800+0000", 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.975800+0000", 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.975799+0000", 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977071+0000", 2024-09-16T06:48:42.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001272537 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977089+0000", 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7988000000000001e-05 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977100+0000", 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0750999999999999e-05 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977132+0000", 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2286999999999998e-05 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977189+0000", 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6771000000000002e-05 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.977931+0000", 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00074231899999999996 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.980412+0000", 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002481249 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.987684+0000", 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0072718000000000001 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.987855+0000", 2024-09-16T06:48:42.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017083800000000001 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:243 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.899579614, 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01277557, 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.974 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 243 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981977+0000", 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981982+0000", 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0669999999999999e-06 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981993+0000", 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0535e-05 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.982010+0000", 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7210999999999998e-05 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.984848+0000", 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028378570000000001 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.984935+0000", 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7505999999999999e-05 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989994+0000", 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050582639999999998 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990209+0000", 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021580699999999999 2024-09-16T06:48:42.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994753+0000", 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045440899999999998 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:285 3.4s0 3:29582896:::smithi11731613-28:6 [read 0~458752,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.007628+0000", 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.873929298, 2024-09-16T06:48:42.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011462415, 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 285 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007628+0000", 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007628+0000", 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007627+0000", 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007635+0000", 2024-09-16T06:48:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6930000000000007e-06 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007653+0000", 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7788e-05 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007674+0000", 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0925999999999999e-05 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.011982+0000", 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043087990000000003 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012289+0000", 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030681099999999999 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.019091+0000", 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068014649999999996 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.983 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:288 3.4s0 3:2be2ffb2:::smithi11731613-4:head [rollback 6] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.103294+0000", 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.778263109999999, 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01131003, 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 288 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103294+0000", 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103294+0000", 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103293+0000", 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103302+0000", 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1719999999999996e-06 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103321+0000", 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814999999999999e-05 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103334+0000", 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3003000000000001e-05 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.106066+0000", 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027321429999999998 2024-09-16T06:48:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.106345+0000", 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027896300000000002 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108016+0000", 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016707989999999999 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.110649+0000", 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026328110000000001 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.114434+0000", 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037854910000000002 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.114604+0000", 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017000000000000001 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.989 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:289 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.77814989, 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013340714, 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:42.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 289 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103406+0000", 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103414+0000", 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7219999999999999e-06 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103437+0000", 2024-09-16T06:48:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2785e-05 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103449+0000", 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2629000000000001e-05 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108874+0000", 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054250820000000003 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:42.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.109046+0000", 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000171173 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.109782+0000", 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073599699999999995 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.111135+0000", 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013535349999999999 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.116590+0000", 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054547659999999998 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.116748+0000", 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015846100000000001 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs5 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:48:42.995 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:39.858276+0000", 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.023281361, 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011228286000000001, 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858276+0000", 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:42.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858276+0000", 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858276+0000", 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858310+0000", 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4261999999999998e-05 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858406+0000", 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6336999999999996e-05 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858434+0000", 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7141e-05 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858557+0000", 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000123433 2024-09-16T06:48:42.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858583+0000", 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6101999999999998e-05 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.869445+0000", 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010861911 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:42.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.869504+0000", 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9487000000000002e-05 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.000 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:43.002 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:48:43.002 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:42.999+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:164: done (0 left) 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:165: delete oid 26 current snap is 14 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:165: done (0 left) 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 18 current snap is 14 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:166: seq_num 89 ranges {294912=737280,1032192=573440,1605632=196608} 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:166: writing smithi11731613-18 from 294912 to 1032192 tid 1 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:166: writing smithi11731613-18 from 1032192 to 1605632 tid 2 2024-09-16T06:48:43.003 INFO:tasks.rados.rados.0.smithi117.stdout:166: writing smithi11731613-18 from 1605632 to 1802240 tid 3 2024-09-16T06:48:43.004 INFO:tasks.rados.rados.0.smithi117.stdout:167: delete oid 27 current snap is 14 2024-09-16T06:48:43.004 INFO:tasks.rados.rados.0.smithi117.stdout:166: finishing write tid 1 to smithi11731613-18 2024-09-16T06:48:43.004 INFO:tasks.rados.rados.0.smithi117.stdout:167: done (1 left) 2024-09-16T06:48:43.004 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 44 current snap is 14 2024-09-16T06:48:43.004 INFO:tasks.rados.rados.0.smithi117.stdout:168: seq_num 91 ranges {1802240=770048,2572288=475136,3047424=294912} 2024-09-16T06:48:43.009 INFO:tasks.rados.rados.0.smithi117.stdout:168: writing smithi11731613-44 from 1802240 to 2572288 tid 1 2024-09-16T06:48:43.014 INFO:tasks.rados.rados.0.smithi117.stdout:168: writing smithi11731613-44 from 2572288 to 3047424 tid 2 2024-09-16T06:48:43.018 INFO:tasks.rados.rados.0.smithi117.stdout:168: writing smithi11731613-44 from 3047424 to 3342336 tid 3 2024-09-16T06:48:43.018 INFO:tasks.rados.rados.0.smithi117.stdout:169: snap_remove snap 10 2024-09-16T06:48:43.102 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:43.099+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:43.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:43.177 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:43.177 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:43.177 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:43.177 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:43.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:48:43.203 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:43.203+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:43.299 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:43.299 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:43.303 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:43.303+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:43.321 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:43.347 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:43.347 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:43.359 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:43.359 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:43.359 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:43.359 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:43.359 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:43.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:48:43.416 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:43.415+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:43.506 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:43.503+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:43.543 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:43.543 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:43.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:43.543 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:43.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 35/32 ECSubWrite(tid=5, reqid=client.4511.0:50, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.928468+0000", 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.614233293000002, 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004872103, 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.544 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928468+0000", 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928468+0000", 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928466+0000", 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928521+0000", 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4593999999999999e-05 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928615+0000", 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4092999999999995e-05 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928723+0000", 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010840600000000001 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929279+0000", 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055508 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929301+0000", 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2150999999999998e-05 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933291+0000", 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039900680000000003 2024-09-16T06:48:43.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933340+0000", 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9419999999999998e-05 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=1, reqid=client.4511.0:53, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.600097655999999, 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.67331382100000003, 2024-09-16T06:48:43.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942602+0000", 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942641+0000", 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9292999999999998e-05 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614766+0000", 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.67212478200000003 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614782+0000", 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5654999999999998e-05 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614883+0000", 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010158 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614901+0000", 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7337999999999999e-05 2024-09-16T06:48:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.615860+0000", 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00095891399999999995 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.615918+0000", 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7978e-05 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.553 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=157, reqid=client.4511.0:76, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.977358+0000", 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.565343118000001, 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048110519999999997, 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977358+0000", 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977358+0000", 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977356+0000", 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.977962+0000", 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00060570999999999997 2024-09-16T06:48:43.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978057+0000", 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4975999999999993e-05 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978091+0000", 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3804999999999999e-05 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978191+0000", 2024-09-16T06:48:43.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100482 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.978206+0000", 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4752999999999999e-05 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982118+0000", 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039119350000000001 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.982169+0000", 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1134999999999997e-05 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=160, reqid=client.4511.0:119, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.513726231, 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050875180000000001, 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028974+0000", 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029016+0000", 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2521000000000002e-05 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029189+0000", 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017280600000000001 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029213+0000", 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4394999999999999e-05 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029795+0000", 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00058197499999999998 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029816+0000", 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0650999999999999e-05 2024-09-16T06:48:43.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.034007+0000", 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041915119999999997 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.034063+0000", 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5145999999999997e-05 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.562 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=18, reqid=client.4511.0:130, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.497078257999998, 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051140320000000001, 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045622+0000", 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046511+0000", 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00088898600000000003 2024-09-16T06:48:43.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046635+0000", 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124404 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046683+0000", 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.761e-05 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046815+0000", 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013226000000000001 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046828+0000", 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2502e-05 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050686+0000", 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038587869999999998 2024-09-16T06:48:43.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050737+0000", 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0785000000000003e-05 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=19, reqid=client.4511.0:131, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.046717+0000", 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.495984485000001, 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043397979999999997, 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046717+0000", 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046717+0000", 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046716+0000", 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046757+0000", 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.07e-05 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046822+0000", 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5283999999999994e-05 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046834+0000", 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2310000000000001e-05 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047497+0000", 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00066222799999999995 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.047512+0000", 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5651e-05 2024-09-16T06:48:43.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051012+0000", 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034997539999999999 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051057+0000", 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4702e-05 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=20, reqid=client.4511.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.491043501, 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061083889999999997, 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051650+0000", 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999924 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051678+0000", 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8297999999999999e-05 2024-09-16T06:48:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051750+0000", 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1749999999999996e-05 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051777+0000", 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7169999999999999e-05 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051852+0000", 2024-09-16T06:48:43.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4768e-05 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051866+0000", 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3893000000000001e-05 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057731+0000", 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00586551 2024-09-16T06:48:43.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057766+0000", 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4795999999999997e-05 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=25, reqid=client.4511.0:140, at_version=35'6, trim_to=0'0, roll_forward_to=35'3))", 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.061567+0000", 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.481134832999999, 2024-09-16T06:48:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047379930000000002, 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061567+0000", 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061567+0000", 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061566+0000", 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061587+0000", 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656000000000001e-05 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061663+0000", 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5850000000000001e-05 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.061671+0000", 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0800000000000006e-06 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.062587+0000", 2024-09-16T06:48:43.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00091552599999999995 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.062608+0000", 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1171000000000001e-05 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.066282+0000", 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036744220000000001 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.066305+0000", 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2220000000000001e-05 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=31, reqid=client.4511.0:161, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.027493124999999, 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054402210000000003, 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515206+0000", 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515247+0000", 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0228999999999997e-05 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515342+0000", 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5420000000000005e-05 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515356+0000", 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3952e-05 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516560+0000", 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012035839999999999 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516583+0000", 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3064000000000001e-05 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520594+0000", 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040113630000000004 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520648+0000", 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4425000000000002e-05 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:221 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:43.585 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.579040166, 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0098559149999999998, 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 221 2024-09-16T06:48:43.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963659+0000", 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963670+0000", 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0423e-05 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963685+0000", 2024-09-16T06:48:43.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5596000000000001e-05 2024-09-16T06:48:43.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963698+0000", 2024-09-16T06:48:43.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2413e-05 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963842+0000", 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144444 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963862+0000", 2024-09-16T06:48:43.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9874999999999999e-05 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.967541+0000", 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036789790000000002 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.967631+0000", 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9548000000000004e-05 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.973517+0000", 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00588654 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:222 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.578983361000001, 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010055573, 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 222 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963717+0000", 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963728+0000", 2024-09-16T06:48:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0609e-05 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963739+0000", 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0829000000000001e-05 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963762+0000", 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2894000000000001e-05 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024621499999999999 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964013+0000", 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7840000000000003e-06 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968066+0000", 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040535229999999998 2024-09-16T06:48:43.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968114+0000", 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7788999999999999e-05 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.973774+0000", 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056596190000000003 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:223 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.578896941, 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011053090999999999, 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 223 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963803+0000", 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963818+0000", 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4983999999999999e-05 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963830+0000", 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1472e-05 2024-09-16T06:48:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963839+0000", 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2420000000000006e-06 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964057+0000", 2024-09-16T06:48:43.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217455 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964066+0000", 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9570000000000008e-06 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968321+0000", 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042553460000000001 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968427+0000", 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105887 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974857+0000", 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064307560000000001 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.603 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 40/37 ECSubWrite(tid=21, reqid=client.4511.0:219, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.552353198, 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057451949999999998, 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990346+0000", 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990377+0000", 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0463999999999998e-05 2024-09-16T06:48:43.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990469+0000", 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1932e-05 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990492+0000", 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3127999999999999e-05 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990628+0000", 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000136212 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990653+0000", 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5289000000000001e-05 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.996040+0000", 2024-09-16T06:48:43.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053864309999999997 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.996093+0000", 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3735e-05 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 40/37 ECSubWrite(tid=59, reqid=client.4511.0:233, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:43.608 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.548979224, 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064058910000000004, 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:43.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993721+0000", 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994930+0000", 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001209803 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995211+0000", 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028033799999999998 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995239+0000", 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8453999999999999e-05 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995377+0000", 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000137797 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995407+0000", 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0154999999999998e-05 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000083+0000", 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046756920000000004 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000128+0000", 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5241999999999998e-05 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:286 3.bs0 3:db3fccb6:::smithi11731613-6:head [stat,delete] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.535268467, 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019136058000000001, 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:43.613 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 286 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:43.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007430+0000", 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007447+0000", 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7146000000000002e-05 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007483+0000", 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-05 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007508+0000", 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5701e-05 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007675+0000", 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016656 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007699+0000", 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3742000000000001e-05 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012471+0000", 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047720949999999996 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012760+0000", 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028969899999999998 2024-09-16T06:48:43.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.014682+0000", 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019216789999999999 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.017646+0000", 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029635830000000001 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.026466+0000", 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0088207990000000007 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.026569+0000", 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010246099999999999 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:39.858539+0000", 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6841618990000002, 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051563729999999997, 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858539+0000", 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858539+0000", 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858538+0000", 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858605+0000", 2024-09-16T06:48:43.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7175999999999995e-05 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858706+0000", 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010049 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858754+0000", 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7854e-05 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858912+0000", 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000158243 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858940+0000", 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7915e-05 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.863635+0000", 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046950519999999999 2024-09-16T06:48:43.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.863696+0000", 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0782999999999997e-05 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 52/37 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'37, trim_to=0'0, roll_forward_to=50'35))", 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:39.883404+0000", 2024-09-16T06:48:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6592969719999999, 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043007230000000002, 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883404+0000", 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883404+0000", 2024-09-16T06:48:43.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883403+0000", 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883452+0000", 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8912999999999997e-05 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883688+0000", 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023581200000000001 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883706+0000", 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8342000000000002e-05 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883828+0000", 2024-09-16T06:48:43.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000121296 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.883853+0000", 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5669000000000001e-05 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.887648+0000", 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037951479999999999 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.887705+0000", 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6626000000000003e-05 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:426 3.bs0 3:db3fccb6:::smithi11731613-6:head [copy-from ver 2] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:40.856438+0000", 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 2.6862635510000001, 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.63165375099999999, 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:43.631 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 426 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856438+0000", 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856438+0000", 2024-09-16T06:48:43.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856435+0000", 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856451+0000", 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6209e-05 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856487+0000", 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5737e-05 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856507+0000", 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0489e-05 2024-09-16T06:48:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856665+0000", 2024-09-16T06:48:43.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015755399999999999 2024-09-16T06:48:43.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:43.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856691+0000", 2024-09-16T06:48:43.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5955000000000001e-05 2024-09-16T06:48:43.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864103+0000", 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074121389999999999 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864254+0000", 2024-09-16T06:48:43.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015063000000000001 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.484545+0000", 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.62029161200000005 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.487326+0000", 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002780827 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.487444+0000", 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118403 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.488092+0000", 2024-09-16T06:48:43.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00064706500000000005 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:427 3.bs0 3:db3fccb6:::smithi11731613-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:40.856528+0000", 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 2.6861732489999999, 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.62975156700000001, 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:43.640 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 427 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856528+0000", 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856528+0000", 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856527+0000", 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856534+0000", 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1749999999999999e-06 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856556+0000", 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2469000000000001e-05 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856574+0000", 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7805999999999999e-05 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856805+0000", 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000230669 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856818+0000", 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3106e-05 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864540+0000", 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077218599999999997 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864630+0000", 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0366999999999997e-05 2024-09-16T06:48:43.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.485781+0000", 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.621151224 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.486017+0000", 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000235436 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.486280+0000", 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000262828 2024-09-16T06:48:43.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 55/37 ECSubWrite(tid=123, reqid=client.4511.0:439, at_version=55'42, trim_to=0'0, roll_forward_to=52'39))", 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:43.016089+0000", 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.52661189100000005, 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004459434, 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:43.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.016089+0000", 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.016089+0000", 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.016089+0000", 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.016992+0000", 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00090269899999999997 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.017084+0000", 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1802e-05 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.017107+0000", 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2935e-05 2024-09-16T06:48:43.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.017256+0000", 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000149822 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.017286+0000", 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9974000000000001e-05 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.020504+0000", 2024-09-16T06:48:43.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003217985 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:43.020549+0000", 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4576999999999997e-05 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:43.650 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:43.651 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:43.652 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:43.653 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:43.653 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:43.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:48:43.655 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:43.647+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:43.659 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:43.659 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:43.696 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:43.732 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:43.732 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:43.753 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:43.752+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:43.765 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:43.766 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:43.781 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:43.781 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:43.781 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:43.781 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:43.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:48:43.801 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:43.801 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:43.836 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:43.837 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:43.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:43.836+0000 7f74e6ffd640 0 -- 172.21.15.100:0/2280952618 >> [v2:172.21.15.117:6810/3208528083,v1:172.21.15.117:6811/3208528083] conn(0x7f74e820ec80 msgr2=0x7f74e8211070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-16T06:48:43.853 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:43.851+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:43.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:48:43.880 INFO:tasks.rados.rados.0.smithi117.stdout:169: done (2 left) 2024-09-16T06:48:43.880 INFO:tasks.rados.rados.0.smithi117.stdout:170: snap_remove snap 7 2024-09-16T06:48:43.922 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:43.922 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:43.936 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:48:43.954 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:43.951+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:44.055 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:44.051+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:44.056 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.056 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.063 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:44.064 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:44.070 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:48:44.087 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:44.087 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:44.112 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:44.112 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:44.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:44.112+0000 7f74e6ffd640 0 -- 172.21.15.100:0/2280952618 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7f74e82181b0 msgr2=0x7f74e821a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:44.156 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:44.155+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:44.191 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.191 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.205 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:48:44.257 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:44.256+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:44.326 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.326 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.341 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:48:44.348 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:44.349 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:44.357 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:44.356+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:44.392 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:44.392 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:44.459 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:44.459+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:44.459 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.459 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.474 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:48:44.559 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:44.556+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:44.605 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.605 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:48:44.660 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:44.655+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:44.737 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.738 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:48:44.760 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:44.760+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:44.861 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:44.864+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:44.871 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:44.871 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:44.881 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:48:44.884 INFO:tasks.rados.rados.0.smithi117.stdout:170: done (2 left) 2024-09-16T06:48:44.884 INFO:tasks.rados.rados.0.smithi117.stdout:171: copy_from oid 30 from oid 37 current snap is 14 2024-09-16T06:48:44.885 INFO:tasks.rados.rados.0.smithi117.stdout:172: snap_create 2024-09-16T06:48:44.885 INFO:tasks.rados.rados.0.smithi117.stdout:166: finishing write tid 2 to smithi11731613-18 2024-09-16T06:48:44.885 INFO:tasks.rados.rados.0.smithi117.stdout:166: finishing write tid 3 to smithi11731613-18 2024-09-16T06:48:44.885 INFO:tasks.rados.rados.0.smithi117.stdout:166: finishing write tid 4 to smithi11731613-18 2024-09-16T06:48:44.885 INFO:tasks.rados.rados.0.smithi117.stdout:168: finishing write tid 1 to smithi11731613-44 2024-09-16T06:48:44.885 INFO:tasks.rados.rados.0.smithi117.stdout:168: finishing write tid 2 to smithi11731613-44 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: finishing write tid 3 to smithi11731613-44 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: finishing write tid 4 to smithi11731613-44 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: finishing write tid 5 to smithi11731613-44 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: oid 44 updating version 0 to 45 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: oid 44 version 45 is already newer than 44 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: oid 44 version 45 is already newer than 42 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: oid 44 version 45 is already newer than 43 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 45 (ObjNum 91 snap 14 seq_num 91) dirty exists 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:168: left oid 44 (ObjNum 91 snap 14 seq_num 91) 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:166: finishing write tid 5 to smithi11731613-18 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:166: oid 18 updating version 0 to 5 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:166: oid 18 updating version 5 to 6 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:166: oid 18 updating version 6 to 7 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:166: oid 18 version 7 is already newer than 4 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 18 v 7 (ObjNum 89 snap 14 seq_num 89) dirty exists 2024-09-16T06:48:44.886 INFO:tasks.rados.rados.0.smithi117.stdout:166: left oid 18 (ObjNum 89 snap 14 seq_num 89) 2024-09-16T06:48:44.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:48:44.915 INFO:tasks.rados.rados.0.smithi117.stdout:171: finishing copy_from racing read to smithi11731613-30 2024-09-16T06:48:44.931 INFO:tasks.rados.rados.0.smithi117.stdout:171: finishing copy_from to smithi11731613-30 2024-09-16T06:48:44.931 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 19 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-16T06:48:44.962 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:44.960+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:45.011 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.011 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.024 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:48:45.063 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:45.064+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:45.148 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.148 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.162 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:48:45.163 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:45.163+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:45.264 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:45.263+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:45.278 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.278 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:48:45.365 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:45.363+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:45.412 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.412 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.426 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:48:45.487 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:45.471+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:45.549 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.549 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.563 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:48:45.568 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:45.568+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:45.670 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:45.672+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:45.720 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.720 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:48:45.771 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:45.767+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:45.861 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:45.861 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:45.871 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:45.871+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:45.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:48:45.900 INFO:tasks.rados.rados.0.smithi117.stdout:166: done (3 left) 2024-09-16T06:48:45.900 INFO:tasks.rados.rados.0.smithi117.stdout:168: done (2 left) 2024-09-16T06:48:45.900 INFO:tasks.rados.rados.0.smithi117.stdout:171: done (1 left) 2024-09-16T06:48:45.900 INFO:tasks.rados.rados.0.smithi117.stdout:172: done (0 left) 2024-09-16T06:48:45.900 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 37 current snap is 15 2024-09-16T06:48:45.900 INFO:tasks.rados.rados.0.smithi117.stdout:173: seq_num 92 ranges {606208=786432,1392640=491520,1884160=278528} 2024-09-16T06:48:45.919 INFO:tasks.rados.rados.0.smithi117.stdout:173: writing smithi11731613-37 from 606208 to 1392640 tid 1 2024-09-16T06:48:45.926 INFO:tasks.rados.rados.0.smithi117.stdout:173: writing smithi11731613-37 from 1392640 to 1884160 tid 2 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:173: writing smithi11731613-37 from 1884160 to 2162688 tid 3 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:174: read oid 47 snap 8 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:174: expect (ObjNum 59 snap 3 seq_num 59) 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:175: read oid 35 snap 14 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:175: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 44 current snap is 15 2024-09-16T06:48:45.931 INFO:tasks.rados.rados.0.smithi117.stdout:176: seq_num 93 ranges {3342336=802816,4145152=638976,4784128=131072} 2024-09-16T06:48:45.972 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:45.972+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:45.977 INFO:tasks.rados.rados.0.smithi117.stdout:176: writing smithi11731613-44 from 3342336 to 4145152 tid 1 2024-09-16T06:48:45.983 INFO:tasks.rados.rados.0.smithi117.stdout:176: writing smithi11731613-44 from 4145152 to 4784128 tid 2 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:176: writing smithi11731613-44 from 4784128 to 4915200 tid 3 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:177: snap_create 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:173: finishing write tid 1 to smithi11731613-37 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:173: finishing write tid 2 to smithi11731613-37 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:173: finishing write tid 3 to smithi11731613-37 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:173: finishing write tid 4 to smithi11731613-37 2024-09-16T06:48:45.985 INFO:tasks.rados.rados.0.smithi117.stdout:175: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-16T06:48:46.003 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:46.004 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:46.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:173: finishing write tid 5 to smithi11731613-37 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:173: oid 37 updating version 0 to 30 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:173: oid 37 updating version 30 to 32 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:173: oid 37 version 32 is already newer than 29 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:173: oid 37 version 32 is already newer than 31 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 32 (ObjNum 92 snap 15 seq_num 92) dirty exists 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:173: left oid 37 (ObjNum 92 snap 15 seq_num 92) 2024-09-16T06:48:46.029 INFO:tasks.rados.rados.0.smithi117.stdout:174: expect (ObjNum 59 snap 3 seq_num 59) 2024-09-16T06:48:46.073 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:46.075+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:46.091 INFO:tasks.rados.rados.0.smithi117.stdout:176: finishing write tid 1 to smithi11731613-44 2024-09-16T06:48:46.091 INFO:tasks.rados.rados.0.smithi117.stdout:176: finishing write tid 2 to smithi11731613-44 2024-09-16T06:48:46.091 INFO:tasks.rados.rados.0.smithi117.stdout:176: finishing write tid 3 to smithi11731613-44 2024-09-16T06:48:46.091 INFO:tasks.rados.rados.0.smithi117.stdout:176: finishing write tid 4 to smithi11731613-44 2024-09-16T06:48:46.091 INFO:tasks.rados.rados.0.smithi117.stdout:176: finishing write tid 5 to smithi11731613-44 2024-09-16T06:48:46.091 INFO:tasks.rados.rados.0.smithi117.stdout:176: oid 44 updating version 0 to 62 2024-09-16T06:48:46.092 INFO:tasks.rados.rados.0.smithi117.stdout:176: oid 44 version 62 is already newer than 60 2024-09-16T06:48:46.092 INFO:tasks.rados.rados.0.smithi117.stdout:176: oid 44 version 62 is already newer than 59 2024-09-16T06:48:46.092 INFO:tasks.rados.rados.0.smithi117.stdout:176: oid 44 version 62 is already newer than 61 2024-09-16T06:48:46.092 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 62 (ObjNum 93 snap 15 seq_num 93) dirty exists 2024-09-16T06:48:46.092 INFO:tasks.rados.rados.0.smithi117.stdout:176: left oid 44 (ObjNum 93 snap 15 seq_num 93) 2024-09-16T06:48:46.134 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:46.134 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:46.148 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:48:46.174 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:46.171+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:46.275 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:46.271+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:46.280 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:46.280 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:46.294 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:48:46.381 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:46.379+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:46.417 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:46.461 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:46.461 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:46.475 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:48:46.477 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:46.476+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:46.578 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:46.575+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:46.579 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:46.579 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:46.603 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:46.604 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:46.629 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:46.629 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:46.643 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:46.643 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:46.657 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:48:46.657 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:46.679 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:46.679+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:46.681 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:46.681 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:46.711 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:46.711 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:46.739 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:46.739 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:46.769 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:46.769 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:46.780 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:46.775+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:46.800 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:46.800 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:173: done (4 left) 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:174: done (3 left) 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:175: done (2 left) 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:176: done (1 left) 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:177: done (0 left) 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:178: read oid 21 snap 9 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:178: expect (ObjNum 59 snap 3 seq_num 59) 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:179: copy_from oid 16 from oid 36 current snap is 16 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:180: read oid 39 snap 14 2024-09-16T06:48:46.818 INFO:tasks.rados.rados.0.smithi117.stdout:180: expect (ObjNum 70 snap 6 seq_num 70) 2024-09-16T06:48:46.819 INFO:tasks.rados.rados.0.smithi117.stdout:181: setattr oid 2 current snap is 16 2024-09-16T06:48:46.822 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 19 current snap is 16 2024-09-16T06:48:46.822 INFO:tasks.rados.rados.0.smithi117.stdout:182: seq_num 94 ranges {1687552=425984,2113536=786432,2899968=376832} 2024-09-16T06:48:46.828 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:46.828 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:46.828 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:46.828 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:46.836 INFO:tasks.rados.rados.0.smithi117.stdout:182: writing smithi11731613-19 from 1687552 to 2113536 tid 1 2024-09-16T06:48:46.837 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:46.837 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:46.842 INFO:tasks.rados.rados.0.smithi117.stdout:182: writing smithi11731613-19 from 2113536 to 2899968 tid 2 2024-09-16T06:48:46.845 INFO:tasks.rados.rados.0.smithi117.stdout:182: writing smithi11731613-19 from 2899968 to 3276800 tid 3 2024-09-16T06:48:46.846 INFO:tasks.rados.rados.0.smithi117.stdout:183: snap_create 2024-09-16T06:48:46.847 INFO:tasks.rados.rados.0.smithi117.stdout:178: expect (ObjNum 59 snap 3 seq_num 59) 2024-09-16T06:48:46.854 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:48:46.866 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:46.866 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:46.866 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T06:48:46.866 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T06:48:46.871 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:46.871 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:46.880 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:46.880+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:46.890 INFO:tasks.rados.rados.0.smithi117.stdout:179: finishing copy_from racing read to smithi11731613-16 2024-09-16T06:48:46.890 INFO:tasks.rados.rados.0.smithi117.stdout:182: finishing write tid 1 to smithi11731613-19 2024-09-16T06:48:46.890 INFO:tasks.rados.rados.0.smithi117.stdout:182: finishing write tid 2 to smithi11731613-19 2024-09-16T06:48:46.890 INFO:tasks.rados.rados.0.smithi117.stdout:182: finishing write tid 3 to smithi11731613-19 2024-09-16T06:48:46.890 INFO:tasks.rados.rados.0.smithi117.stdout:182: finishing write tid 4 to smithi11731613-19 2024-09-16T06:48:46.890 INFO:tasks.rados.rados.0.smithi117.stdout:180: expect (ObjNum 70 snap 6 seq_num 70) 2024-09-16T06:48:46.894 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:46.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:48:46.915 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:46.915 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:46.941 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:46.941 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:46.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:46.944+0000 7fdca0de9640 0 -- 172.21.15.100:0/709838301 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7fdc9c21ee40 msgr2=0x7fdc9c221230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 2 v 68 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:182: finishing write tid 5 to smithi11731613-19 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:182: oid 19 updating version 0 to 65 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:182: oid 19 updating version 65 to 66 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:182: oid 19 version 66 is already newer than 63 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:182: oid 19 version 66 is already newer than 64 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 19 v 66 (ObjNum 94 snap 16 seq_num 94) dirty exists 2024-09-16T06:48:46.947 INFO:tasks.rados.rados.0.smithi117.stdout:182: left oid 19 (ObjNum 94 snap 16 seq_num 94) 2024-09-16T06:48:46.970 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:46.971 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:46.981 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:46.979+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:47.011 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:47.012 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:47.012 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:47.012 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:47.012 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:47.026 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:48:47.082 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:47.080+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:1 3.ds0 3:b273fc7d:::smithi11731613-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.257772580000001, 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011215246, 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 1 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892973+0000", 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893050+0000", 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6996999999999995e-05 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893075+0000", 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5406999999999999e-05 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893092+0000", 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6541999999999998e-05 2024-09-16T06:48:47.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893183+0000", 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1404999999999996e-05 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893334+0000", 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015069399999999999 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894090+0000", 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00075619499999999998 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901851+0000", 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0077611449999999997 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012789380000000001 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904191+0000", 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001060636 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.257638615000001, 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.159 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:48:47.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.164 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.257554818999999, 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:48:47.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:47.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:48:47.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:48:47.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.226149580000001, 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:48:47.172 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:48:47.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.177 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.226099812000001, 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:48:47.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:48:47.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:48:47.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.118362146999999, 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.183 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:47.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:48:47.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:48:47.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.188 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.117806688000002, 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.189 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:48:47.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:47.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.117750356999998, 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:48:47.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:219 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [copy-from ver 4] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.194531207000001, 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031057211000000001, 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.200 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 219 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:48:47.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956214+0000", 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956234+0000", 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0275000000000002e-05 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956284+0000", 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0219999999999997e-05 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956309+0000", 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4930999999999999e-05 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956442+0000", 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013270999999999999 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:47.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956459+0000", 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7317999999999999e-05 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960744+0000", 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0042850090000000002 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961009+0000", 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026485300000000001 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.976584+0000", 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015575011999999999 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.981658+0000", 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050738470000000003 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987101+0000", 2024-09-16T06:48:47.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054435580000000003 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987274+0000", 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017305700000000001 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:220 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:48:47.206 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.194408749000001, 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023931746, 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 220 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:48:47.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956338+0000", 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956351+0000", 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2849e-05 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956375+0000", 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4195000000000001e-05 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956391+0000", 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5352000000000001e-05 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956650+0000", 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025920099999999997 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956669+0000", 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9157999999999999e-05 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961818+0000", 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051493709999999998 2024-09-16T06:48:47.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961948+0000", 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012956 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979661+0000", 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017713421 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979887+0000", 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022550399999999999 2024-09-16T06:48:47.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.980271+0000", 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038454100000000002 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 40/32 ECSubWrite(tid=28, reqid=client.4511.0:226, at_version=0'0, trim_to=0'0, roll_forward_to=40'19))", 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.985513+0000", 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.165235382999999, 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0097361930000000006, 2024-09-16T06:48:47.212 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985513+0000", 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985513+0000", 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985511+0000", 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985541+0000", 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9677999999999999e-05 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985646+0000", 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105235 2024-09-16T06:48:47.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985671+0000", 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5335000000000001e-05 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.986750+0000", 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001078608 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.986775+0000", 2024-09-16T06:48:47.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5327999999999999e-05 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995186+0000", 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084108599999999992 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995249+0000", 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3151999999999994e-05 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.216 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 42/37 ECSubWrite(tid=6, reqid=client.4511.0:286, at_version=42'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.013094+0000", 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 17.137654475000001, 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066263540000000001, 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.013094+0000", 2024-09-16T06:48:47.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.013094+0000", 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.013093+0000", 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.013157+0000", 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3866000000000001e-05 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.014077+0000", 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092024399999999997 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.014100+0000", 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2371999999999999e-05 2024-09-16T06:48:47.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.014225+0000", 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000125208 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.014250+0000", 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4675e-05 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.019645+0000", 2024-09-16T06:48:47.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0053949810000000001 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.019720+0000", 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5548000000000002e-05 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:316 3.ds0 3:b2c197ab:::smithi11731613-17:head [copy-from ver 4] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 16.988733245999999, 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017020408000000001, 2024-09-16T06:48:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 316 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162012+0000", 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162030+0000", 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8688000000000001e-05 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162067+0000", 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6526000000000002e-05 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162083+0000", 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6061000000000001e-05 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162190+0000", 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107351 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162511+0000", 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032124399999999999 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.174275+0000", 2024-09-16T06:48:47.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011763691999999999 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.176204+0000", 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019286329999999999 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.178962+0000", 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027583099999999999 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.179036+0000", 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3456999999999998e-05 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:317 3.ds0 3:b2c197ab:::smithi11731613-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 16.988647911000001, 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013682241, 2024-09-16T06:48:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 317 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162106+0000", 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4620000000000001e-06 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162124+0000", 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7706999999999998e-05 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162136+0000", 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2285e-05 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162962+0000", 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000826123 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163038+0000", 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5371999999999996e-05 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175269+0000", 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01223169 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175524+0000", 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025445300000000003 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175783+0000", 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025890899999999998 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:352 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842404+0000", 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.308344511, 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01003983, 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 352 2024-09-16T06:48:47.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842404+0000", 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842404+0000", 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842400+0000", 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842421+0000", 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0271999999999998e-05 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842447+0000", 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6692000000000001e-05 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842462+0000", 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4676e-05 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842520+0000", 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7707000000000002e-05 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842621+0000", 2024-09-16T06:48:47.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000100756 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.852444+0000", 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0098231069999999993 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:353 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:48:47.237 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.308266282, 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011292597, 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 353 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842481+0000", 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842490+0000", 2024-09-16T06:48:47.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4760000000000004e-06 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842509+0000", 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8924999999999999e-05 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842523+0000", 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4446e-05 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.843021+0000", 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000497589 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.843096+0000", 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5332000000000003e-05 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853775+0000", 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010678309 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.241 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:354 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.308210353, 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013614067000000001, 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 354 2024-09-16T06:48:47.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842537+0000", 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842545+0000", 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2500000000000006e-06 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842562+0000", 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7512000000000001e-05 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842573+0000", 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0976e-05 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844555+0000", 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001981919 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844621+0000", 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5792999999999998e-05 2024-09-16T06:48:47.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.856152+0000", 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011530883 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/37 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.846380+0000", 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.304368456000001, 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0074597700000000001, 2024-09-16T06:48:47.246 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.846380+0000", 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.846380+0000", 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.846379+0000", 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847105+0000", 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00072596300000000004 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850412+0000", 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0033067159999999999 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850422+0000", 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0081999999999999e-05 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.852556+0000", 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021343970000000001 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853840+0000", 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001283392 2024-09-16T06:48:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/37 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.847119+0000", 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.303629527, 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0068832110000000002, 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847119+0000", 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847119+0000", 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847118+0000", 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847640+0000", 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00052163399999999999 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850445+0000", 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028049139999999999 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850454+0000", 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3950000000000007e-06 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853899+0000", 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034450779999999999 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.854002+0000", 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010305 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 49/32 ECSubWrite(tid=55, reqid=osd.10.0:55, at_version=49'38, trim_to=0'0, roll_forward_to=47'34))", 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.836348+0000", 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.314400633, 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0068904989999999996, 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836348+0000", 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836348+0000", 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836346+0000", 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836406+0000", 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0553000000000001e-05 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836491+0000", 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5208000000000001e-05 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836517+0000", 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6018999999999999e-05 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836632+0000", 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011497000000000001 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836655+0000", 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2795000000000001e-05 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.843169+0000", 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065135510000000002 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.843238+0000", 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9472000000000002e-05 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.258 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:47.261 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:48:47.261 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:47.261 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:47.261 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 59, 2024-09-16T06:48:47.261 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "b2976256-637c-4e28-94f6-99f20017bc5e", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "created": "2024-09-16T06:47:53.157733+0000", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "modified": "2024-09-16T06:48:46.805721+0000", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "last_up_change": "2024-09-16T06:48:06.616578+0000", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "last_in_change": "2024-09-16T06:48:24.894403+0000", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_num": 5806080, 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_set": [ 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "nodeep-scrub", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "noscrub", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "pglog_hardlimit", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "purged_snapdirs", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_deletes", 2024-09-16T06:48:47.262 INFO:teuthology.orchestra.run.smithi100.stdout: "sortbitwise" 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_version": 6, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_snapshot": "", 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_max": 3, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd": 16, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "require_min_compat_client": "luminous", 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "min_compat_client": "jewel", 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "require_osd_release": "reef", 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_crimson": false, 2024-09-16T06:48:47.263 INFO:teuthology.orchestra.run.smithi100.stdout: "pools": [ 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 1, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": ".mgr", 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:05.838641+0000", 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:47.264 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 1, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 1, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 1, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 1, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 1, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:47.265 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "25", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:47.266 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:47.267 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:47.268 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_max": 32, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 1 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": {} 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 15.380000114440918, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 15.380000114440918, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.12999999523162842, 2024-09-16T06:48:47.269 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 0.97786140441894531, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "rbd", 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:08.489111+0000", 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:47.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 8, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 8, 2024-09-16T06:48:47.271 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 8, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 8, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 8, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "28", 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:47.272 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 3, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 28, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:47.273 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:47.274 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd": {} 2024-09-16T06:48:47.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 4.0199999809265137, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 4.0199999809265137, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.56000000238418579, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2.25, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2.25, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 0.97786140441894531, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:47.276 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "unique_pool_0", 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:18.885917+0000", 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 3, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 7, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 5, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 1, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:47.277 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 16, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 16, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 16, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 16, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 16, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:47.278 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "59", 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 25, 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 59, 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:47.279 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:47.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "shecprofile", 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:47.281 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 16384, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "rados": {} 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osds": [ 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "54494294-f25f-41f2-bff7-106492665e07", 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.282 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6808", 2024-09-16T06:48:47.283 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6809", 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.284 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6810", 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6811", 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6814", 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6815", 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6812", 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6813", 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.287 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6809/1028275949", 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6811/1028275949", 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6815/1028275949", 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6813/1028275949", 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "a265e752-1cd6-497e-bcb9-c56591a5ce30", 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.288 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.289 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6808", 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6809", 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6811", 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6813", 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6819", 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6821", 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.292 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6815", 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6817", 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6809/930331367", 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6813/930331367", 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6821/930331367", 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6817/930331367", 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "dada34c6-4dc6-4f9f-bc31-85983fd2d7fd", 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.294 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6800", 2024-09-16T06:48:47.295 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6801", 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6802", 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6803", 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6806", 2024-09-16T06:48:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6807", 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6804", 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6805", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6801/4231506893", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6803/4231506893", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6807/4231506893", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6805/4231506893", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "6c0fc051-a4a0-4ce5-989a-668e75bbe3c0", 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 54, 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6802", 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6803", 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6804", 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6805", 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6808", 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6809", 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6806", 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6807", 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6803/1995864012", 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6805/1995864012", 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6809/1995864012", 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6807/1995864012", 2024-09-16T06:48:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "36a46387-b859-4937-bbd0-5dd708630c6e", 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6802", 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6803", 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6806", 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6807", 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6804", 2024-09-16T06:48:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6805", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6801/2054405220", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6803/2054405220", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6807/2054405220", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6805/2054405220", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "073f3d23-77f0-4fd3-b067-27fa81137288", 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6810", 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6812", 2024-09-16T06:48:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6814", 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6816", 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6822", 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6823", 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6818", 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6820", 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6812/1081990569", 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6816/1081990569", 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6823/1081990569", 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6820/1081990569", 2024-09-16T06:48:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "8a81592f-3704-48f6-89ba-cc4997c2105e", 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6824", 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6825", 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6826", 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6827", 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6830", 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6831", 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6828", 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6829", 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6825/1095047955", 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6827/1095047955", 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6831/1095047955", 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6829/1095047955", 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27", 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 0.645782470703125, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6818", 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6819", 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6820", 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6821", 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6824", 2024-09-16T06:48:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6825", 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6822", 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6823", 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6819/3607859516", 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6821/3607859516", 2024-09-16T06:48:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6825/3607859516", 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6823/3607859516", 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7ce38131-057f-4d45-88a5-dae8bcceae3c", 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6816", 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6817", 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6818", 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6819", 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6822", 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6823", 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6820", 2024-09-16T06:48:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6821", 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6817/232055141", 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6819/232055141", 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6823/232055141", 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6821/232055141", 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "aa7fde6e-6563-4e37-8af2-9114d0091362", 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6824", 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6825", 2024-09-16T06:48:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6826", 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6827", 2024-09-16T06:48:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6830", 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6831", 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6828", 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6829", 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6825/1511881207", 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6827/1511881207", 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6831/1511881207", 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6829/1511881207", 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "05c5389d-8082-4be3-aa6d-138e00720f06", 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6808", 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6809", 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6810", 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6811", 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6814", 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6815", 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6812", 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6813", 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6809/435900412", 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6811/435900412", 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6815/435900412", 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6813/435900412", 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "99552590-1d0a-420e-b0c3-357246880658", 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6810", 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6811", 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6812", 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6813", 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6816", 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6817", 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6814", 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6815", 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6811/3208528083", 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6813/3208528083", 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6817/3208528083", 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6815/3208528083", 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "165fea1c-74b6-4806-a0d9-47441e161ed5", 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6824", 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6825", 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6826", 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6827", 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6830", 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6831", 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6828", 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6829", 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6825/3107910928", 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6827/3107910928", 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6831/3107910928", 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6829/3107910928", 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "03402d0c-e7fd-4233-b2cd-bf79cd75e9bc", 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 0, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 0, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6800", 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6801", 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6802", 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6803", 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6806", 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6807", 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6804", 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6805", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6801/316232366", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6803/316232366", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6807/316232366", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6805/316232366", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7b027faf-5c9e-4aee-9cb9-9c6459b8fe39", 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6816", 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6817", 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6818", 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6819", 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6822", 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6823", 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6820", 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6821", 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6817/693497656", 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6819/693497656", 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6823/693497656", 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6821/693497656", 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "4877aa40-c0d8-494f-8a98-36089ed694c2", 2024-09-16T06:48:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6826", 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6827", 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6828", 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6829", 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6832", 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6833", 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6830", 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6831", 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6827/579912865", 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6829/579912865", 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6833/579912865", 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6831/579912865", 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_xinfo": [ 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.584963+0000", 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.677023+0000", 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.608639+0000", 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.384489+0000", 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.480551+0000", 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.476690+0000", 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.615096+0000", 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.581224+0000", 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.597357+0000", 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.549502+0000", 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.448485+0000", 2024-09-16T06:48:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.495825+0000", 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.628407+0000", 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 65536, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.368452+0000", 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.600045+0000", 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.567594+0000", 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap": [], 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_items": [], 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_primaries": [], 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_temp": [], 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_temp": [], 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklist": {}, 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "range_blocklist": {}, 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profiles": { 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "default": { 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "2", 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "1", 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "jerasure", 2024-09-16T06:48:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "reed_sol_van" 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "shecprofile": { 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "c": "2", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-device-class": "", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-root": "default", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "4", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "3", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "shecprofile", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "shec", 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "multiple" 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps_queue": [ 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 11, 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 14, 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 19, 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "new_removed_snaps": [], 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "new_purged_snaps": [ 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 6, 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 2 2024-09-16T06:48:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 9, 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_node_flags": {}, 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "device_class_flags": {}, 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode": { 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_enabled": false, 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_bucket_count": 0, 2024-09-16T06:48:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "degraded_stretch_mode": 0, 2024-09-16T06:48:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "recovering_stretch_mode": 0, 2024-09-16T06:48:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_bucket": 0 2024-09-16T06:48:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:47.397 INFO:tasks.thrashosds.thrasher:j is {'epoch': 59, 'fsid': 'b2976256-637c-4e28-94f6-99f20017bc5e', 'created': '2024-09-16T06:47:53.157733+0000', 'modified': '2024-09-16T06:48:46.805721+0000', 'last_up_change': '2024-09-16T06:48:06.616578+0000', 'last_in_change': '2024-09-16T06:48:24.894403+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T06:48:05.838641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9778614044189453, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T06:48:08.489111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9778614044189453, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-16T06:48:18.885917+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 59, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '54494294-f25f-41f2-bff7-106492665e07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6808', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6809', 'nonce': 1028275949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6810', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6811', 'nonce': 1028275949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6814', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6815', 'nonce': 1028275949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6812', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6813', 'nonce': 1028275949}]}, 'public_addr': '172.21.15.100:6809/1028275949', 'cluster_addr': '172.21.15.100:6811/1028275949', 'heartbeat_back_addr': '172.21.15.100:6815/1028275949', 'heartbeat_front_addr': '172.21.15.100:6813/1028275949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a265e752-1cd6-497e-bcb9-c56591a5ce30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 930331367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6811', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 930331367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6819', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 930331367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6815', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 930331367}]}, 'public_addr': '172.21.15.107:6809/930331367', 'cluster_addr': '172.21.15.107:6813/930331367', 'heartbeat_back_addr': '172.21.15.107:6821/930331367', 'heartbeat_front_addr': '172.21.15.107:6817/930331367', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'dada34c6-4dc6-4f9f-bc31-85983fd2d7fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 4231506893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 4231506893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 4231506893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 4231506893}]}, 'public_addr': '172.21.15.115:6801/4231506893', 'cluster_addr': '172.21.15.115:6803/4231506893', 'heartbeat_back_addr': '172.21.15.115:6807/4231506893', 'heartbeat_front_addr': '172.21.15.115:6805/4231506893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6c0fc051-a4a0-4ce5-989a-668e75bbe3c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6802', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6803', 'nonce': 1995864012}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6804', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6805', 'nonce': 1995864012}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6808', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6809', 'nonce': 1995864012}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6806', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6807', 'nonce': 1995864012}]}, 'public_addr': '172.21.15.117:6803/1995864012', 'cluster_addr': '172.21.15.117:6805/1995864012', 'heartbeat_back_addr': '172.21.15.117:6809/1995864012', 'heartbeat_front_addr': '172.21.15.117:6807/1995864012', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '36a46387-b859-4937-bbd0-5dd708630c6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 2054405220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6802', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6803', 'nonce': 2054405220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6806', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6807', 'nonce': 2054405220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6804', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6805', 'nonce': 2054405220}]}, 'public_addr': '172.21.15.100:6801/2054405220', 'cluster_addr': '172.21.15.100:6803/2054405220', 'heartbeat_back_addr': '172.21.15.100:6807/2054405220', 'heartbeat_front_addr': '172.21.15.100:6805/2054405220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '073f3d23-77f0-4fd3-b067-27fa81137288', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6812', 'nonce': 1081990569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6816', 'nonce': 1081990569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 1081990569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6820', 'nonce': 1081990569}]}, 'public_addr': '172.21.15.107:6812/1081990569', 'cluster_addr': '172.21.15.107:6816/1081990569', 'heartbeat_back_addr': '172.21.15.107:6823/1081990569', 'heartbeat_front_addr': '172.21.15.107:6820/1081990569', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8a81592f-3704-48f6-89ba-cc4997c2105e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 1095047955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1095047955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1095047955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1095047955}]}, 'public_addr': '172.21.15.115:6825/1095047955', 'cluster_addr': '172.21.15.115:6827/1095047955', 'heartbeat_back_addr': '172.21.15.115:6831/1095047955', 'heartbeat_front_addr': '172.21.15.115:6829/1095047955', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.645782470703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6818', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6819', 'nonce': 3607859516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6820', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6821', 'nonce': 3607859516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6824', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6825', 'nonce': 3607859516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6822', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6823', 'nonce': 3607859516}]}, 'public_addr': '172.21.15.117:6819/3607859516', 'cluster_addr': '172.21.15.117:6821/3607859516', 'heartbeat_back_addr': '172.21.15.117:6825/3607859516', 'heartbeat_front_addr': '172.21.15.117:6823/3607859516', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '7ce38131-057f-4d45-88a5-dae8bcceae3c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6816', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6817', 'nonce': 232055141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6818', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6819', 'nonce': 232055141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6822', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6823', 'nonce': 232055141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6820', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6821', 'nonce': 232055141}]}, 'public_addr': '172.21.15.100:6817/232055141', 'cluster_addr': '172.21.15.100:6819/232055141', 'heartbeat_back_addr': '172.21.15.100:6823/232055141', 'heartbeat_front_addr': '172.21.15.100:6821/232055141', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'aa7fde6e-6563-4e37-8af2-9114d0091362', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 1511881207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 1511881207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 1511881207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 1511881207}]}, 'public_addr': '172.21.15.107:6825/1511881207', 'cluster_addr': '172.21.15.107:6827/1511881207', 'heartbeat_back_addr': '172.21.15.107:6831/1511881207', 'heartbeat_front_addr': '172.21.15.107:6829/1511881207', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '05c5389d-8082-4be3-aa6d-138e00720f06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 435900412}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 435900412}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 435900412}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 435900412}]}, 'public_addr': '172.21.15.115:6809/435900412', 'cluster_addr': '172.21.15.115:6811/435900412', 'heartbeat_back_addr': '172.21.15.115:6815/435900412', 'heartbeat_front_addr': '172.21.15.115:6813/435900412', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '99552590-1d0a-420e-b0c3-357246880658', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6810', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6811', 'nonce': 3208528083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6812', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6813', 'nonce': 3208528083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6816', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6817', 'nonce': 3208528083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6814', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6815', 'nonce': 3208528083}]}, 'public_addr': '172.21.15.117:6811/3208528083', 'cluster_addr': '172.21.15.117:6813/3208528083', 'heartbeat_back_addr': '172.21.15.117:6817/3208528083', 'heartbeat_front_addr': '172.21.15.117:6815/3208528083', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '165fea1c-74b6-4806-a0d9-47441e161ed5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6824', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6825', 'nonce': 3107910928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6826', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6827', 'nonce': 3107910928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6830', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6831', 'nonce': 3107910928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6828', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6829', 'nonce': 3107910928}]}, 'public_addr': '172.21.15.100:6825/3107910928', 'cluster_addr': '172.21.15.100:6827/3107910928', 'heartbeat_back_addr': '172.21.15.100:6831/3107910928', 'heartbeat_front_addr': '172.21.15.100:6829/3107910928', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '03402d0c-e7fd-4233-b2cd-bf79cd75e9bc', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 316232366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 316232366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 316232366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 316232366}]}, 'public_addr': '172.21.15.107:6801/316232366', 'cluster_addr': '172.21.15.107:6803/316232366', 'heartbeat_back_addr': '172.21.15.107:6807/316232366', 'heartbeat_front_addr': '172.21.15.107:6805/316232366', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '7b027faf-5c9e-4aee-9cb9-9c6459b8fe39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 693497656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 693497656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 693497656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 693497656}]}, 'public_addr': '172.21.15.115:6817/693497656', 'cluster_addr': '172.21.15.115:6819/693497656', 'heartbeat_back_addr': '172.21.15.115:6823/693497656', 'heartbeat_front_addr': '172.21.15.115:6821/693497656', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '4877aa40-c0d8-494f-8a98-36089ed694c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6826', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6827', 'nonce': 579912865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6828', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6829', 'nonce': 579912865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 579912865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6830', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6831', 'nonce': 579912865}]}, 'public_addr': '172.21.15.117:6827/579912865', 'cluster_addr': '172.21.15.117:6829/579912865', 'heartbeat_back_addr': '172.21.15.117:6833/579912865', 'heartbeat_front_addr': '172.21.15.117:6831/579912865', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.584963+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.677023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.608639+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.384489+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.480551+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.476690+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.615096+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.581224+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.597357+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.549502+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.448485+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.495825+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.628407+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.368452+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.600045+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.567594+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 11, 'length': 1}, {'begin': 14, 'length': 1}, {'begin': 19, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 6, 'length': 2}, {'begin': 9, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T06:48:47.397 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T06:48:47.398 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:47.255+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:47.399 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:47.399+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:47.499 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:47.495+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:47.530 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:47.530 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:47.530 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:47.530 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:47.544 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:48:47.600 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:47.599+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:47.663 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:47.663 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:47.663 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:47.677 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:48:47.701 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:47.700+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:47.752 INFO:tasks.rados.rados.0.smithi117.stdout:179: finishing copy_from to smithi11731613-16 2024-09-16T06:48:47.752 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 16 v 4 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-16T06:48:47.767 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:47.768 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:15 3.fs0 3:fe6e9dcb:::smithi11731613-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895424+0000", 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.899099681999999, 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033755443000000003, 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 15 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895424+0000", 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895424+0000", 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895423+0000", 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895428+0000", 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1500000000000001e-06 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895438+0000", 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0497e-05 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895445+0000", 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2660000000000004e-06 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898920+0000", 2024-09-16T06:48:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034744609999999999 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.899038+0000", 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000118103 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.923251+0000", 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024213373999999999 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.801 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.923421+0000", 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016966799999999999 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.929180+0000", 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0057586740000000001 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.802 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:19 3.fs0 3:fa1c58b4:::smithi11731613-7:head [append 0~114688,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895453+0000", 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.899071085999999, 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031813735000000003, 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 19 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.803 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895453+0000", 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895453+0000", 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895452+0000", 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895633+0000", 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000180905 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895647+0000", 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3987e-05 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895657+0000", 2024-09-16T06:48:47.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0040999999999999e-05 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.899080+0000", 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034227849999999998 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.899249+0000", 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016941699999999999 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.900154+0000", 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090474399999999997 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911262+0000", 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011108386 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926372+0000", 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015110006 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927266+0000", 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00089414599999999996 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:20 3.fs0 3:fa1c58b4:::smithi11731613-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895667+0000", 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.898857029999999, 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034325209000000002, 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.808 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 20 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895667+0000", 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895667+0000", 2024-09-16T06:48:47.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895666+0000", 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895671+0000", 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9010000000000002e-06 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.810 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895681+0000", 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9760000000000007e-06 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895693+0000", 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2534000000000001e-05 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909215+0000", 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013521348000000001 2024-09-16T06:48:47.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.909779+0000", 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00056460899999999999 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.910929+0000", 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011494649999999999 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913071+0000", 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021418029999999998 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.929931+0000", 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016859986 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.929992+0000", 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.1470999999999995e-05 2024-09-16T06:48:47.813 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:21 3.fs0 3:fa1c58b4:::smithi11731613-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.898823141000001, 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.036267527000000001, 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.814 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 21 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895700+0000", 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895704+0000", 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.048e-06 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895714+0000", 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8509999999999994e-06 2024-09-16T06:48:47.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895721+0000", 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2339999999999997e-06 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913213+0000", 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017491642000000002 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913291+0000", 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8053999999999995e-05 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930092+0000", 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016801100999999999 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930264+0000", 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017232 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.931968+0000", 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017039620000000001 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:25 3.fs0 3:f3961c35:::smithi11731613-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.898795331999999, 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.036948371000000001, 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.819 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 25 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:47.820 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895853+0000", 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000125451 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895874+0000", 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0346000000000001e-05 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895883+0000", 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0999999999999993e-06 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913349+0000", 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017466612999999999 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913577+0000", 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022744 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914548+0000", 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00097095700000000005 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917700+0000", 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031521740000000002 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.932545+0000", 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014845191000000001 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.932677+0000", 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013169899999999999 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:26 3.fs0 3:f3961c35:::smithi11731613-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.824 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.898631945999998, 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.037776441000000001, 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 26 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:48:47.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895891+0000", 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895896+0000", 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9599999999999999e-06 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895906+0000", 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9969999999999993e-06 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895913+0000", 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.08e-06 2024-09-16T06:48:47.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915339+0000", 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.019426282 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915437+0000", 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7822e-05 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916103+0000", 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00066603099999999998 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917703+0000", 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015995969999999999 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.933600+0000", 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015897047000000001 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.933668+0000", 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8465000000000003e-05 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:27 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.898603057999999, 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039993954999999998, 2024-09-16T06:48:47.830 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 27 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895920+0000", 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895924+0000", 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.506e-06 2024-09-16T06:48:47.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895933+0000", 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.876e-06 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895941+0000", 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8229999999999995e-06 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916469+0000", 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.020528105000000001 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916546+0000", 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6815000000000002e-05 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.934316+0000", 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017769672 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.934388+0000", 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1753000000000002e-05 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.935915+0000", 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015271079999999999 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.835 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 35/32 ECSubWrite(tid=35, reqid=client.4511.0:152, at_version=35'10, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.716684097000002, 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.44133388099999998, 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077839+0000", 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077850+0000", 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0505e-05 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515878+0000", 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.43802848300000002 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515883+0000", 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1780000000000002e-06 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515923+0000", 2024-09-16T06:48:47.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9866000000000001e-05 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515930+0000", 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.224e-06 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519096+0000", 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031664449999999999 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519174+0000", 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7590999999999996e-05 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:163 3.cs0 3:31c9f1c0:::smithi11731613-46:head [copy-from ver 2 in=54b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.282442407000001, 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.80778171600000004, 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.840 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 163 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512085+0000", 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8870000000000002e-06 2024-09-16T06:48:47.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512091+0000", 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3539999999999996e-06 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512096+0000", 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8029999999999996e-06 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513808+0000", 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001712199 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513857+0000", 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8649000000000001e-05 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.524232+0000", 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010375422 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.527189+0000", 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002956939 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319694+0000", 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79250512900000003 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319863+0000", 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168796 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.845 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.267815337999998, 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79400328899999995, 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526711+0000", 2024-09-16T06:48:47.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.18e-06 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319361+0000", 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79264922400000004 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319384+0000", 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3442e-05 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.848 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319508+0000", 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123683 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.320712+0000", 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012041 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:48:47.849 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.265840394000001, 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.394952786, 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528687+0000", 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0810000000000004e-06 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923559+0000", 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39487182199999998 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923572+0000", 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3406e-05 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923624+0000", 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1989e-05 2024-09-16T06:48:47.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923636+0000", 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1990999999999999e-05 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:233 3.cs0 3:3e2de01d:::smithi11731613-47:head [copy-from ver 22] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956752+0000", 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.837771384, 2024-09-16T06:48:47.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042655024, 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 233 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956752+0000", 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956752+0000", 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956749+0000", 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:47.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956767+0000", 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8751999999999999e-05 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956819+0000", 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1112999999999997e-05 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956841+0000", 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2410999999999999e-05 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956968+0000", 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127071 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956981+0000", 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2731999999999999e-05 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960674+0000", 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036936619999999999 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960877+0000", 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000202368 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.991730+0000", 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.030852876000000001 2024-09-16T06:48:47.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995444+0000", 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037141629999999999 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.999320+0000", 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0038765150000000001 2024-09-16T06:48:47.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.999407+0000", 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7151000000000006e-05 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:234 3.cs0 3:3e2de01d:::smithi11731613-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956880+0000", 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.837643947, 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038279512000000002, 2024-09-16T06:48:47.860 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 234 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956880+0000", 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956880+0000", 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956878+0000", 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956887+0000", 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3389999999999993e-06 2024-09-16T06:48:47.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956910+0000", 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3587999999999998e-05 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956927+0000", 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6905000000000002e-05 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.957104+0000", 2024-09-16T06:48:47.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017687099999999999 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.957114+0000", 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4849999999999998e-06 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961213+0000", 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0040992349999999997 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:47.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961336+0000", 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123201 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.994816+0000", 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033479832000000001 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995000+0000", 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018455000000000001 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995159+0000", 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015890199999999999 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=97, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.949724234, 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15568405900000001, 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.866 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844798+0000", 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844807+0000", 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8319999999999995e-06 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.998993+0000", 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15418552099999999 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999006+0000", 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3069e-05 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999149+0000", 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000142965 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000484+0000", 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013350300000000001 2024-09-16T06:48:47.869 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 49/37 ECSubWriteReply(tid=98, last_complete=49'26, committed=1, applied=1))", 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.944013311999999, 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15060995699999999, 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.870 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850509+0000", 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:47.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850523+0000", 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4219000000000001e-05 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000937+0000", 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15041373399999999 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000960+0000", 2024-09-16T06:48:47.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3768e-05 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001073+0000", 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112218 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001120+0000", 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7936999999999999e-05 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.873 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as6 49/32 ECSubWrite(tid=80, reqid=client.4511.0:373, at_version=49'61, trim_to=0'0, roll_forward_to=49'59))", 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.862745013, 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.071706169, 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931778+0000", 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931805+0000", 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7250000000000002e-05 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002139+0000", 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.070333802000000001 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002154+0000", 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5176e-05 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002259+0000", 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105357 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002284+0000", 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4743e-05 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.003425+0000", 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001140605 2024-09-16T06:48:47.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.003485+0000", 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0442999999999997e-05 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=99, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.792421296000001, 2024-09-16T06:48:47.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123751468, 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002101+0000", 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002109+0000", 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8399999999999995e-06 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125675+0000", 2024-09-16T06:48:47.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123566417 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125711+0000", 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5215000000000001e-05 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125821+0000", 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110661 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125854+0000", 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2638000000000002e-05 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:428 3.cs0 3:31c9f1c0:::smithi11731613-46:head [delete] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 6.9396220480000004, 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.62638514099999998, 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.882 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 428 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854899+0000", 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854915+0000", 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5262999999999999e-05 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854948+0000", 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3005e-05 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854965+0000", 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7112000000000002e-05 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855070+0000", 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105198 2024-09-16T06:48:47.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855081+0000", 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1308000000000001e-05 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478008+0000", 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.622927122 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478196+0000", 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000187534 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.481287+0000", 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030910360000000001 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(osd.5.22:1 3.fs0 3:fa1c58b4:::smithi11731613-7:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:48:47.887 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 6.9258438079999998, 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.61158470300000001, 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868678+0000", 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868690+0000", 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2336e-05 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868722+0000", 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2122000000000002e-05 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868744+0000", 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1855e-05 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868876+0000", 2024-09-16T06:48:47.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013210300000000001 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868888+0000", 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2053e-05 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478066+0000", 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.60917769200000005 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478207+0000", 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000140743 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.480265+0000", 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020579219999999998 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:431 3.fs0 3:f3961c35:::smithi11731613-9:head [stat,delete] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:48:47.892 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.9795436730000002, 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.106237357, 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 431 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:47.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814977+0000", 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814993+0000", 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6095000000000001e-05 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815025+0000", 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1822000000000001e-05 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815042+0000", 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6803000000000002e-05 2024-09-16T06:48:47.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815181+0000", 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013884600000000001 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815482+0000", 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030067499999999998 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:47.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.817743+0000", 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022618769999999998 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.820149+0000", 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024051290000000002 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921014+0000", 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.10086516700000001 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:47.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921218+0000", 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020381300000000001 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:47.898 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:47.900 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:48:47.901 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:48:47.901 INFO:tasks.rados.rados.0.smithi117.stdout:178: done (5 left) 2024-09-16T06:48:47.901 INFO:tasks.rados.rados.0.smithi117.stdout:179: done (4 left) 2024-09-16T06:48:47.901 INFO:tasks.rados.rados.0.smithi117.stdout:180: done (3 left) 2024-09-16T06:48:47.901 INFO:tasks.rados.rados.0.smithi117.stdout:181: done (2 left) 2024-09-16T06:48:47.901 INFO:tasks.rados.rados.0.smithi117.stdout:182: done (1 left) 2024-09-16T06:48:47.902 INFO:tasks.rados.rados.0.smithi117.stdout:183: done (0 left) 2024-09-16T06:48:47.902 INFO:tasks.rados.rados.0.smithi117.stdout:184: snap_remove snap 9 2024-09-16T06:48:47.902 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:47.899+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:48.000 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:47.999+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:48.069 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:48.070 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:48.070 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:48.084 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:48:48.101 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:48.099+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:48.202 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:48.199+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:48.205 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:48.205 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:48.205 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:48.206 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:48.206 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:48.219 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:48:48.303 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:48.300+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:48.344 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:48.344 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:48.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:48.344 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:48.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.344 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.897946+0000", 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.444134528999999, 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0069738930000000001, 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897946+0000", 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897946+0000", 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897945+0000", 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.897973+0000", 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8302000000000001e-05 2024-09-16T06:48:48.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898056+0000", 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2748000000000001e-05 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898068+0000", 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2767e-05 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898146+0000", 2024-09-16T06:48:48.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7732999999999996e-05 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898159+0000", 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3233e-05 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904857+0000", 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066979099999999996 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904920+0000", 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2213999999999998e-05 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:85 3.5s0 3:af79444c:::smithi11731613-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.983710+0000", 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.358370542999999, 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.006079175, 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.349 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 85 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.983710+0000", 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.983710+0000", 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.983708+0000", 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984127+0000", 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00041826 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984139+0000", 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2675e-05 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984147+0000", 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0989999999999999e-06 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984216+0000", 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8924000000000006e-05 2024-09-16T06:48:48.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984374+0000", 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015792200000000001 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.985825+0000", 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001450331 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988717+0000", 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028918870000000001 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989713+0000", 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00099672999999999993 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989789+0000", 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5542000000000003e-05 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.354 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:86 3.5s0 3:af79444c:::smithi11731613-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.984167+0000", 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.357912978000002, 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0059986040000000003, 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 86 2024-09-16T06:48:48.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984167+0000", 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984167+0000", 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984166+0000", 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984171+0000", 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0669999999999999e-06 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984178+0000", 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3239999999999999e-06 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984182+0000", 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5290000000000002e-06 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.986462+0000", 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022802090000000001 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.986519+0000", 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6132999999999999e-05 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.987064+0000", 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00054519200000000001 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988721+0000", 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001657438 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990112+0000", 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013912169999999999 2024-09-16T06:48:48.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990166+0000", 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3412000000000003e-05 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:87 3.5s0 3:af79444c:::smithi11731613-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.984219+0000", 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.357861236000002, 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084208679999999998, 2024-09-16T06:48:48.360 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 87 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984219+0000", 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984219+0000", 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984218+0000", 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984223+0000", 2024-09-16T06:48:48.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3919999999999996e-06 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984232+0000", 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4870000000000008e-06 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.984239+0000", 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7399999999999998e-06 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.987296+0000", 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030572870000000001 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.987421+0000", 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000124315 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990433+0000", 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030123810000000002 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.990508+0000", 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5105999999999995e-05 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992640+0000", 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002131834 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.365 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 40/32 ECSubWrite(tid=43, reqid=client.4511.0:253, at_version=40'31, trim_to=0'0, roll_forward_to=39'29))", 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.087470+0000", 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.254609863999999, 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0055944690000000004, 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.087470+0000", 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.087470+0000", 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.087470+0000", 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.087808+0000", 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00033827800000000002 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.087939+0000", 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013045900000000001 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.087966+0000", 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7713e-05 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.088124+0000", 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000157582 2024-09-16T06:48:48.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.088149+0000", 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5038999999999999e-05 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.092999+0000", 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048502730000000004 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.093065+0000", 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5499000000000004e-05 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:266 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.388251330999999, 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014774862999999999, 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:48.370 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 266 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953825+0000", 2024-09-16T06:48:48.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953846+0000", 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0171999999999999e-05 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953889+0000", 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3284000000000001e-05 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953913+0000", 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4071e-05 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954055+0000", 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014240699999999999 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954076+0000", 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0678000000000001e-05 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961148+0000", 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070716959999999997 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961402+0000", 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025470100000000003 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.968604+0000", 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072013340000000002 2024-09-16T06:48:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:267 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.388135119000001, 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015485804000000001, 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:48.375 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 267 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953944+0000", 2024-09-16T06:48:48.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953973+0000", 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9581e-05 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953995+0000", 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1685e-05 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954011+0000", 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5862000000000002e-05 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954223+0000", 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021187199999999999 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954233+0000", 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0789e-05 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962540+0000", 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0083068819999999998 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962710+0000", 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017007799999999999 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.969431+0000", 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0067204250000000004 2024-09-16T06:48:48.379 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:268 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.388054872000001, 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.30266284300000001, 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 268 2024-09-16T06:48:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954024+0000", 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954032+0000", 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4290000000000004e-06 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954050+0000", 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8332e-05 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954063+0000", 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2955999999999999e-05 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954379+0000", 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031616000000000001 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954389+0000", 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3959999999999995e-06 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963428+0000", 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0090396959999999998 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963723+0000", 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00029431800000000001 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:29.256688+0000", 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.29296564200000003 2024-09-16T06:48:48.384 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 42/32 ECSubWrite(tid=32, reqid=client.4511.0:309, at_version=42'19, trim_to=0'0, roll_forward_to=42'18))", 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162338+0000", 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.179742628, 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0056838920000000003, 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.385 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162338+0000", 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162338+0000", 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162335+0000", 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:48.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162669+0000", 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00033371699999999998 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162772+0000", 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010295599999999999 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162796+0000", 2024-09-16T06:48:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3832000000000002e-05 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163009+0000", 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021272199999999999 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163036+0000", 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7645e-05 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.167972+0000", 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0049361520000000001 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.168021+0000", 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9030999999999999e-05 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.389 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 47/32 ECSubWrite(tid=47, reqid=client.4511.0:347, at_version=47'31, trim_to=0'0, roll_forward_to=44'29))", 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.833692+0000", 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 13.508388097999999, 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0059672409999999999, 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.833692+0000", 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.833692+0000", 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.833691+0000", 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.834191+0000", 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000499703 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.834269+0000", 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8004999999999996e-05 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.834292+0000", 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3011999999999999e-05 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.834431+0000", 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013925799999999999 2024-09-16T06:48:48.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.834457+0000", 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5709000000000001e-05 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.839596+0000", 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051392010000000004 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.839659+0000", 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3542999999999998e-05 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 47/32 ECSubWrite(tid=49, reqid=client.4511.0:348, at_version=47'32, trim_to=0'0, roll_forward_to=47'31))", 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.844750+0000", 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 13.497330526000001, 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0078023529999999997, 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.394 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844750+0000", 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844750+0000", 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844748+0000", 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847501+0000", 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027531830000000002 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847619+0000", 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011817899999999999 2024-09-16T06:48:48.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847644+0000", 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5329000000000001e-05 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847899+0000", 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025474699999999999 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847926+0000", 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6735999999999999e-05 2024-09-16T06:48:48.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.852497+0000", 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0045709180000000002 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.852552+0000", 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5254000000000003e-05 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 47/32 ECSubWrite(tid=65, reqid=client.4511.0:359, at_version=47'45, trim_to=0'0, roll_forward_to=44'43))", 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.859347+0000", 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 13.482732989000001, 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054040119999999997, 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859347+0000", 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859347+0000", 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859347+0000", 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859390+0000", 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3526000000000003e-05 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859479+0000", 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8758000000000006e-05 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859500+0000", 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1478e-05 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859650+0000", 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014903799999999999 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.859677+0000", 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7078999999999999e-05 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.864700+0000", 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005023408 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.864751+0000", 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1196999999999998e-05 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=51, reqid=osd.14.0:51, at_version=49'36, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.833722+0000", 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.50835797, 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0063840750000000003, 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.403 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.833722+0000", 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.833722+0000", 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.833722+0000", 2024-09-16T06:48:48.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.833775+0000", 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3594000000000002e-05 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.833878+0000", 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000102734 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.833900+0000", 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2311999999999999e-05 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834044+0000", 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014333799999999999 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834067+0000", 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3638999999999998e-05 2024-09-16T06:48:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.840045+0000", 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0059779550000000001 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.840106+0000", 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0992999999999997e-05 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 49/32 ECSubWrite(tid=54, reqid=osd.10.0:54, at_version=49'36, trim_to=0'0, roll_forward_to=47'34))", 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.834235+0000", 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.507844952999999, 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0055732849999999999, 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834235+0000", 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834235+0000", 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834235+0000", 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834284+0000", 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9657000000000002e-05 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834385+0000", 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000100955 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834407+0000", 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2055000000000001e-05 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834519+0000", 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011176299999999999 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.834545+0000", 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5826e-05 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.839620+0000", 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050749669999999997 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.839809+0000", 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018863500000000001 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.506792683, 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014247134, 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835286+0000", 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835349+0000", 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3258999999999999e-05 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835459+0000", 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110097 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835480+0000", 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1424999999999999e-05 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835600+0000", 2024-09-16T06:48:48.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011922 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835623+0000", 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3144999999999999e-05 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849454+0000", 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013830779 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849535+0000", 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1021999999999998e-05 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 49/32 ECSubWrite(tid=55, reqid=osd.10.0:55, at_version=49'38, trim_to=0'0, roll_forward_to=47'34))", 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.836929+0000", 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.505151222, 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011498985999999999, 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.416 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836929+0000", 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836929+0000", 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836927+0000", 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836980+0000", 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2917000000000002e-05 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.837073+0000", 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.2454000000000006e-05 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.837095+0000", 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2322000000000001e-05 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.837221+0000", 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012585 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.837334+0000", 2024-09-16T06:48:48.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011311600000000001 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.848358+0000", 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011024654 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.848428+0000", 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9479000000000003e-05 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.420 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 49/32 ECSubWrite(tid=70, reqid=osd.3.0:70, at_version=49'51, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.856964+0000", 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.485116485000001, 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0053983920000000001, 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856964+0000", 2024-09-16T06:48:48.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856964+0000", 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856962+0000", 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.857012+0000", 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0464000000000003e-05 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.857132+0000", 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000120073 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.857157+0000", 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4805000000000001e-05 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.857300+0000", 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000143182 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.857326+0000", 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5598000000000001e-05 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.862302+0000", 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0049763129999999996 2024-09-16T06:48:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.862362+0000", 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9846999999999998e-05 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=54, reqid=osd.14.0:54, at_version=49'40, trim_to=0'0, roll_forward_to=49'38))", 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.858754+0000", 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.483326395000001, 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070779220000000004, 2024-09-16T06:48:48.425 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.858754+0000", 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.858754+0000", 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.858752+0000", 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.858807+0000", 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5407e-05 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859109+0000", 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030223199999999999 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.427 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859136+0000", 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6502000000000001e-05 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859244+0000", 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000108353 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859267+0000", 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2348999999999999e-05 2024-09-16T06:48:48.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.865760+0000", 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0064937629999999996 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.865832+0000", 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1309000000000005e-05 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.429 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=55, reqid=osd.14.0:55, at_version=49'42, trim_to=0'0, roll_forward_to=49'38))", 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.859622+0000", 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.482457992000001, 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0071865690000000003, 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.430 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859622+0000", 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859622+0000", 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859621+0000", 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859671+0000", 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0037999999999998e-05 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859771+0000", 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9734000000000004e-05 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.859792+0000", 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1019e-05 2024-09-16T06:48:48.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.860252+0000", 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00046000300000000001 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.860275+0000", 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3351999999999999e-05 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.866761+0000", 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0064859560000000002 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.866809+0000", 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7877000000000003e-05 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 56/32 ECSubWrite(tid=103, reqid=osd.3.0:100, at_version=0'0, trim_to=0'0, roll_forward_to=56'81))", 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:43.911203+0000", 2024-09-16T06:48:48.434 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.4308773500000003, 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0064157099999999998, 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911203+0000", 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911203+0000", 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:48.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911200+0000", 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911232+0000", 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1288999999999998e-05 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911342+0000", 2024-09-16T06:48:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011084800000000001 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911371+0000", 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8581999999999998e-05 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911527+0000", 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015628199999999999 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.911553+0000", 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6009000000000001e-05 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.917551+0000", 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0059976109999999999 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:43.917619+0000", 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7663000000000006e-05 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.439 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:48.439 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:48.439 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:48.440 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:48:48.441 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:48.438+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:48.541 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:48.542+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:48.571 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:48.571 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:48.585 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:48:48.641 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:48.639+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:48.709 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:48.710 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:48.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:48:48.742 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:48.739+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:48.815 INFO:tasks.rados.rados.0.smithi117.stdout:184: done (0 left) 2024-09-16T06:48:48.815 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 37 current snap is 17 2024-09-16T06:48:48.815 INFO:tasks.rados.rados.0.smithi117.stdout:185: seq_num 95 ranges {2162688=442368,2605056=540672,3145728=622592} 2024-09-16T06:48:48.844 INFO:tasks.rados.rados.0.smithi117.stdout:185: writing smithi11731613-37 from 2162688 to 2605056 tid 1 2024-09-16T06:48:48.845 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:48.839+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:48.851 INFO:tasks.rados.rados.0.smithi117.stdout:185: writing smithi11731613-37 from 2605056 to 3145728 tid 2 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:185: writing smithi11731613-37 from 3145728 to 3768320 tid 3 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:186: copy_from oid 43 from oid 22 current snap is 17 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:187: rmattr oid 11 current snap is 17 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:187: done (2 left) 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:188: read oid 8 snap 14 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:188: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:189: snap_create 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:186: finishing copy_from to smithi11731613-43 2024-09-16T06:48:48.862 INFO:tasks.rados.rados.0.smithi117.stdout:186: got expected ENOENT (src dne) 2024-09-16T06:48:48.863 INFO:tasks.rados.rados.0.smithi117.stdout:186: finishing copy_from racing read to smithi11731613-43 2024-09-16T06:48:48.865 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:48.865 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:48.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:70 1.0 e24/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'44, mlcod=24'40)", 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:08.537808+0000", 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.330289653000001, 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059384349999999997, 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537808+0000", 2024-09-16T06:48:48.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537808+0000", 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537807+0000", 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537816+0000", 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1220000000000001e-06 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537868+0000", 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2414000000000003e-05 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537879+0000", 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1209999999999999e-05 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537923+0000", 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3164999999999998e-05 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.537936+0000", 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3864e-05 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.543717+0000", 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057801290000000002 2024-09-16T06:48:48.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:08.543746+0000", 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9717999999999999e-05 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:135 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'93, mlcod=25'89)", 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 39.391759020000002, 2024-09-16T06:48:48.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073860260000000004, 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476337+0000", 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476374+0000", 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7509000000000003e-05 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476453+0000", 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8273000000000001e-05 2024-09-16T06:48:48.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476468+0000", 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476574+0000", 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000106879 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476624+0000", 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9910000000000002e-05 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483640+0000", 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00701565 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483724+0000", 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4535000000000003e-05 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:141 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'99, mlcod=25'91)", 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 39.382255213000001, 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.28011762800000001, 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485841+0000", 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485852+0000", 2024-09-16T06:48:48.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1313e-05 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764267+0000", 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.278414628 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764288+0000", 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.105e-05 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764448+0000", 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016007599999999999 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764487+0000", 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9125999999999998e-05 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765872+0000", 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001384763 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765960+0000", 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7979000000000006e-05 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:160 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'114, mlcod=26'107)", 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.938484283999998, 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061551990000000001, 2024-09-16T06:48:48.879 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929617+0000", 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5430000000000004e-06 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929973+0000", 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035563700000000001 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929979+0000", 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7599999999999999e-06 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.931187+0000", 2024-09-16T06:48:48.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001208081 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.931239+0000", 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2522999999999998e-05 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935719+0000", 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044799219999999999 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935768+0000", 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9113000000000002e-05 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:161 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'115, mlcod=26'107)", 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.938477441000003, 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062540549999999997, 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.884 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929715+0000", 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5147000000000002e-05 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929991+0000", 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027581399999999999 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929997+0000", 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1399999999999997e-06 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932071+0000", 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020736499999999998 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932100+0000", 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9090000000000001e-05 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935772+0000", 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0036722019999999998 2024-09-16T06:48:48.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935874+0000", 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102296 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.889 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:162 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'116, mlcod=26'107)", 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.938378368000002, 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062203090000000003, 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929791+0000", 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1872999999999994e-05 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930051+0000", 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026016400000000001 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930059+0000", 2024-09-16T06:48:48.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4100000000000008e-06 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933018+0000", 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029584709999999998 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933048+0000", 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.029e-05 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935882+0000", 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028339509999999999 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935939+0000", 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7636999999999997e-05 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.894 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:163 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'117, mlcod=26'107)", 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.938300349000002, 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062254820000000001, 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929801+0000", 2024-09-16T06:48:48.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2030000000000002e-06 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930091+0000", 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028969100000000002 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930097+0000", 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1870000000000002e-06 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933566+0000", 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003469663 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933600+0000", 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3293000000000001e-05 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935942+0000", 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023424190000000001 2024-09-16T06:48:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936023+0000", 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0438999999999993e-05 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:164 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'118, mlcod=26'107)", 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.938293905999998, 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062774559999999998, 2024-09-16T06:48:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929803+0000", 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929846+0000", 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3038999999999999e-05 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930109+0000", 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026273799999999999 2024-09-16T06:48:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930115+0000", 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0900000000000001e-06 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934327+0000", 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004212248 2024-09-16T06:48:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934363+0000", 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5576e-05 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936028+0000", 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016651770000000001 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936081+0000", 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2874000000000002e-05 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:165 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'119, mlcod=26'107)", 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.938248162999997, 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0069224810000000003, 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:48:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929854+0000", 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7600000000000002e-06 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930127+0000", 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000273374 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930135+0000", 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.157e-06 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935309+0000", 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051733370000000001 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935340+0000", 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1423000000000001e-05 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936696+0000", 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013555730000000001 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936772+0000", 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6260000000000005e-05 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.8s2 35/32 ECSubWrite(tid=159, reqid=client.4511.0:118, at_version=35'3, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.027251+0000", 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.840846170999999, 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598439, 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027251+0000", 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027251+0000", 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027250+0000", 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027487+0000", 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023623000000000001 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027597+0000", 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011027399999999999 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027627+0000", 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9808e-05 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027701+0000", 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4275000000000003e-05 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.027710+0000", 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.5890000000000003e-06 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.033225+0000", 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005515041 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.033236+0000", 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1001e-05 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=32, reqid=client.4511.0:237, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:48:48.913 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.905972177999999, 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.008178467, 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962124+0000", 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962148+0000", 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3572999999999999e-05 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962287+0000", 2024-09-16T06:48:48.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013903200000000001 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962295+0000", 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1999999999999994e-06 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962430+0000", 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135129 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962474+0000", 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3503e-05 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970216+0000", 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077425230000000003 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7330000000000006e-05 2024-09-16T06:48:48.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=34, reqid=client.4511.0:228, at_version=40'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.905223448000001, 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075127500000000003, 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962872+0000", 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962920+0000", 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7939999999999998e-05 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963011+0000", 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0655999999999994e-05 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963034+0000", 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3782999999999999e-05 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964104+0000", 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010698509999999999 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964127+0000", 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2563000000000001e-05 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061773169999999999 2024-09-16T06:48:48.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970387+0000", 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2680000000000001e-05 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=35, reqid=client.4511.0:238, at_version=40'13, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.904791775, 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00704997, 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963304+0000", 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963342+0000", 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7753000000000003e-05 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963421+0000", 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9549999999999996e-05 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963436+0000", 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4885999999999999e-05 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964389+0000", 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095243900000000002 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964403+0000", 2024-09-16T06:48:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.381e-05 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970320+0000", 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059176259999999996 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970356+0000", 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5525999999999998e-05 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.926 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=36, reqid=client.4511.0:229, at_version=40'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.904283486000001, 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006602887, 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963812+0000", 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963860+0000", 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7182999999999999e-05 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963954+0000", 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3912999999999993e-05 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963975+0000", 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1506e-05 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964846+0000", 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087055200000000002 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964868+0000", 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.243e-05 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970396+0000", 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005528397 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970417+0000", 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0418999999999999e-05 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=37, reqid=client.4511.0:239, at_version=40'14, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.904089534000001, 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070022000000000001, 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.931 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964007+0000", 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964083+0000", 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6262999999999998e-05 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964162+0000", 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8568999999999999e-05 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964181+0000", 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9283000000000002e-05 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965031+0000", 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00084985599999999996 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965056+0000", 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4868999999999999e-05 2024-09-16T06:48:48.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970969+0000", 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059132270000000001 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.971010+0000", 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1285999999999997e-05 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=77, reqid=osd.1.0:77, at_version=52'38, trim_to=0'0, roll_forward_to=50'36))", 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 9.0059450170000002, 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061672749999999998, 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862151+0000", 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862203+0000", 2024-09-16T06:48:48.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2352000000000003e-05 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862291+0000", 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8163000000000003e-05 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862313+0000", 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2255999999999998e-05 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862436+0000", 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000122753 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862462+0000", 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5559999999999999e-05 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868253+0000", 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057913009999999996 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868320+0000", 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6855999999999998e-05 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=80, reqid=osd.1.0:78, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 8.9879665590000002, 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064748710000000001, 2024-09-16T06:48:48.940 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880130+0000", 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880152+0000", 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2322000000000001e-05 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880228+0000", 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6367999999999998e-05 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880245+0000", 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6186999999999999e-05 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880330+0000", 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.5568000000000005e-05 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880347+0000", 2024-09-16T06:48:48.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6528000000000002e-05 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886553+0000", 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006206423 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886606+0000", 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2704999999999998e-05 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/37 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'37, trim_to=0'0, roll_forward_to=50'35))", 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 8.9864201179999998, 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0081001449999999996, 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881676+0000", 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881715+0000", 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8512e-05 2024-09-16T06:48:48.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881760+0000", 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4669999999999998e-05 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881776+0000", 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6371999999999998e-05 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881968+0000", 2024-09-16T06:48:48.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019165999999999999 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881998+0000", 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0332999999999999e-05 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889735+0000", 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077367310000000002 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889778+0000", 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2766999999999999e-05 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=81, reqid=osd.1.0:81, at_version=52'42, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 8.9854387340000006, 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076324610000000001, 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882657+0000", 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882697+0000", 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9900000000000001e-05 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882763+0000", 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5949000000000002e-05 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882779+0000", 2024-09-16T06:48:48.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6118000000000001e-05 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883074+0000", 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000294393 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883098+0000", 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4049999999999998e-05 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890245+0000", 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071470479999999996 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890291+0000", 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6343e-05 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=82, reqid=osd.1.0:82, at_version=52'44, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:48:48.953 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.885013+0000", 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 8.9830842799999999, 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060881939999999999, 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885013+0000", 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885013+0000", 2024-09-16T06:48:48.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885012+0000", 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885052+0000", 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9823999999999997e-05 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885135+0000", 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2915000000000002e-05 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.885151+0000", 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6303999999999998e-05 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886686+0000", 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015355430000000001 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886710+0000", 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4063e-05 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.891058+0000", 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043475199999999997 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.891101+0000", 2024-09-16T06:48:48.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3439000000000002e-05 2024-09-16T06:48:48.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.958 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:48.958 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:48.958 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: finishing write tid 1 to smithi11731613-37 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: finishing write tid 2 to smithi11731613-37 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:188: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: finishing write tid 3 to smithi11731613-37 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: finishing write tid 4 to smithi11731613-37 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: finishing write tid 5 to smithi11731613-37 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: oid 37 updating version 0 to 37 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: oid 37 updating version 37 to 39 2024-09-16T06:48:48.960 INFO:tasks.rados.rados.0.smithi117.stdout:185: oid 37 version 39 is already newer than 36 2024-09-16T06:48:48.961 INFO:tasks.rados.rados.0.smithi117.stdout:185: oid 37 version 39 is already newer than 38 2024-09-16T06:48:48.961 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 39 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-16T06:48:48.961 INFO:tasks.rados.rados.0.smithi117.stdout:185: left oid 37 (ObjNum 95 snap 17 seq_num 95) 2024-09-16T06:48:48.961 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:48:48.962 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:48.954+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:49.060 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:49.060+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:49.089 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:49.089 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:49.089 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:49.089 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:49.103 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:48:49.161 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:49.162+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:49.262 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:49.263+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:49.269 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:49.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:49.269 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:49.269 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:49.269 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:49.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:48:49.363 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:49.363+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/32 ECSubWrite(tid=2, reqid=client.4511.0:19, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.489390865000001, 2024-09-16T06:48:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0095178020000000006, 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914640+0000", 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914681+0000", 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1100000000000003e-05 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914894+0000", 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021241899999999999 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914899+0000", 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.857e-06 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916733+0000", 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018331899999999999 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916758+0000", 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5721999999999999e-05 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924087+0000", 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073282740000000001 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924158+0000", 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.148e-05 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.407 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/32 ECSubWrite(tid=3, reqid=client.4511.0:20, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.489341033999999, 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0095058429999999999, 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914690+0000", 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914700+0000", 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0224000000000001e-05 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914912+0000", 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021219500000000001 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.914929+0000", 2024-09-16T06:48:49.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6424999999999999e-05 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917780+0000", 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851494 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917803+0000", 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3071999999999999e-05 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924166+0000", 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063629560000000003 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924196+0000", 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9638999999999999e-05 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.412 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 35/32 ECSubWrite(tid=4, reqid=client.4511.0:25, at_version=35'5, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.916464+0000", 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.487567592000001, 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077895539999999997, 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916464+0000", 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916464+0000", 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916463+0000", 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916594+0000", 2024-09-16T06:48:49.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000131023 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916641+0000", 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7592000000000001e-05 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.916650+0000", 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0529999999999996e-06 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918332+0000", 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001681788 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918355+0000", 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2289999999999998e-05 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924200+0000", 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005845416 2024-09-16T06:48:49.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924253+0000", 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3173000000000001e-05 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=9, reqid=client.4511.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.950424+0000", 2024-09-16T06:48:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.453607056999999, 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0099245310000000003, 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950424+0000", 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950424+0000", 2024-09-16T06:48:49.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950423+0000", 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.950440+0000", 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7235e-05 2024-09-16T06:48:49.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959533+0000", 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090923519999999997 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959542+0000", 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5980000000000003e-06 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959593+0000", 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0599999999999997e-05 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.959599+0000", 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8869999999999997e-06 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.960327+0000", 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00072782399999999996 2024-09-16T06:48:49.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.960349+0000", 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2076999999999999e-05 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=12, reqid=client.4511.0:68, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.442941075, 2024-09-16T06:48:49.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.041915173, 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961089+0000", 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961108+0000", 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8260000000000001e-05 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001032+0000", 2024-09-16T06:48:49.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.039924492999999998 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001043+0000", 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1063000000000001e-05 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001095+0000", 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1703999999999998e-05 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001102+0000", 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8360000000000003e-06 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002963+0000", 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018614529999999999 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.003005+0000", 2024-09-16T06:48:49.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2182999999999999e-05 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=14, reqid=client.4511.0:98, at_version=35'10, trim_to=0'0, roll_forward_to=35'7))", 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.401867323000001, 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082808355, 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002163+0000", 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002178+0000", 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4431e-05 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084421+0000", 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082243256000000001 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084430+0000", 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8880000000000003e-06 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084484+0000", 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4193000000000001e-05 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084496+0000", 2024-09-16T06:48:49.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1754e-05 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084946+0000", 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000450242 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084972+0000", 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6281000000000001e-05 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=16, reqid=client.4511.0:158, at_version=35'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.319647194000002, 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41790576000000001, 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084383+0000", 2024-09-16T06:48:49.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084401+0000", 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7584999999999999e-05 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500734+0000", 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41633377199999999 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500752+0000", 2024-09-16T06:48:49.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7374e-05 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500862+0000", 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010999500000000001 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500877+0000", 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5437000000000001e-05 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502236+0000", 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001358389 2024-09-16T06:48:49.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502290+0000", 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4076000000000002e-05 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:244 3.7s0 3:e60b6e1b:::smithi11731613-14:head [stat,delete] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.982404+0000", 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.421626891999999, 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086814219999999994, 2024-09-16T06:48:49.436 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 244 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982404+0000", 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982404+0000", 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982392+0000", 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999876 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982419+0000", 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7166e-05 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982460+0000", 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1303000000000001e-05 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982482+0000", 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2116999999999998e-05 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982639+0000", 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015620200000000001 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.982907+0000", 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026842000000000001 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.985003+0000", 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020963589999999999 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.987604+0000", 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026004349999999999 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.990914+0000", 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033104649999999998 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.991086+0000", 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017147500000000001 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:249 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~704512] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.057326+0000", 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.346705320000002, 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010497131999999999, 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 249 2024-09-16T06:48:49.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.057326+0000", 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.057326+0000", 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.057323+0000", 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059497+0000", 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002174175 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059528+0000", 2024-09-16T06:48:49.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0790000000000002e-05 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059544+0000", 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6439999999999998e-05 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059635+0000", 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1281999999999998e-05 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.059809+0000", 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017336200000000001 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.061631+0000", 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018220210000000001 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067293+0000", 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056618830000000004 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067733+0000", 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044024500000000002 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.067823+0000", 2024-09-16T06:48:49.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0100999999999997e-05 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:252 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.340514443, 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.38343603199999998, 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:49.449 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 252 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063515+0000", 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063523+0000", 2024-09-16T06:48:49.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7430000000000002e-06 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063545+0000", 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1310000000000001e-05 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063562+0000", 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7629999999999999e-05 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068202+0000", 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046400269999999997 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068467+0000", 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000264691 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070315+0000", 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018477190000000001 2024-09-16T06:48:49.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070443+0000", 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012795699999999999 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.446953+0000", 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.376510228 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:273 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~819200,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:29.968535+0000", 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.435495923000001, 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191088, 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 273 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968535+0000", 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968535+0000", 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968532+0000", 2024-09-16T06:48:49.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968550+0000", 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8558000000000001e-05 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968579+0000", 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8727000000000001e-05 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968591+0000", 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2113000000000001e-05 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968683+0000", 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1625000000000005e-05 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op must wait for map", 2024-09-16T06:48:49.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968697+0000", 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4331999999999999e-05 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971175+0000", 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024773120000000002 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971271+0000", 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6089999999999996e-05 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.975726+0000", 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044557590000000001 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:274 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~819200,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.435425172999999, 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0080167260000000001, 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 274 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968606+0000", 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968610+0000", 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.51e-06 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968615+0000", 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3759999999999999e-06 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968619+0000", 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.861e-06 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968815+0000", 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019582399999999999 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968824+0000", 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3470000000000005e-06 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971697+0000", 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028724000000000002 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.971791+0000", 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4116000000000005e-05 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.976623+0000", 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048317769999999998 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:275 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~819200,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:29.968674+0000", 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.435357407000001, 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0088331550000000005, 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:49.465 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 275 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968674+0000", 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968674+0000", 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968673+0000", 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968677+0000", 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1590000000000002e-06 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968684+0000", 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9310000000000008e-06 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968689+0000", 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1640000000000003e-06 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968863+0000", 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017429899999999999 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.968872+0000", 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7879999999999995e-06 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.972130+0000", 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032578910000000002 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.972206+0000", 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6421000000000006e-05 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:29.977507+0000", 2024-09-16T06:48:49.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053008990000000004 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 52/37 ECSubWrite(tid=113, reqid=osd.11.0:113, at_version=52'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.861506+0000", 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 9.5425255159999995, 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071674260000000002, 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861506+0000", 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861506+0000", 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861503+0000", 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861557+0000", 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3959999999999998e-05 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861663+0000", 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105312 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861685+0000", 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1877000000000001e-05 2024-09-16T06:48:49.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861883+0000", 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019818099999999999 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.861898+0000", 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4895999999999999e-05 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.867872+0000", 2024-09-16T06:48:49.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059742629999999996 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868673+0000", 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000801244 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs2 52/37 ECSubWrite(tid=117, reqid=osd.11.0:115, at_version=0'0, trim_to=0'0, roll_forward_to=52'54))", 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.879370+0000", 2024-09-16T06:48:49.474 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 9.5246607369999996, 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0085184830000000003, 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879370+0000", 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879370+0000", 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879369+0000", 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879389+0000", 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9582e-05 2024-09-16T06:48:49.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879463+0000", 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3619000000000001e-05 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879479+0000", 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6543999999999999e-05 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879799+0000", 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032016100000000001 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.879825+0000", 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.527e-05 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882406+0000", 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002581131 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.887889+0000", 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054831960000000001 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:482 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~442368] snapc 1a=[1a,19,18,15,11,c,8,5] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.553704207, 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019730733, 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.479 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 482 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:48:49.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850326+0000", 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851134+0000", 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00080810400000000003 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851148+0000", 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3689000000000001e-05 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851153+0000", 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8329999999999999e-06 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851191+0000", 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8087000000000003e-05 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851262+0000", 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0964999999999996e-05 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851984+0000", 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00072110499999999999 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.859456+0000", 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0074722319999999997 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.869982+0000", 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010525806 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870058+0000", 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6180000000000006e-05 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.484 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:483 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~540672] snapc 1a=[1a,19,18,15,11,c,8,5] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.856905+0000", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.54712599399999995, 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013382826, 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 483 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.856905+0000", 2024-09-16T06:48:49.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.856905+0000", 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.856901+0000", 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999957 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.857604+0000", 2024-09-16T06:48:49.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000702968 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.857620+0000", 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5639999999999999e-05 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.857632+0000", 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2037e-05 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.857671+0000", 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9705e-05 2024-09-16T06:48:49.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.857742+0000", 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0647000000000003e-05 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.858417+0000", 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00067487299999999999 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.860848+0000", 2024-09-16T06:48:49.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024314150000000001 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870220+0000", 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0093712299999999995 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870288+0000", 2024-09-16T06:48:49.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8499999999999998e-05 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:484 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~622592] snapc 1a=[1a,19,18,15,11,c,8,5] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.865404+0000", 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.53862742900000005, 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011438874, 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.490 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 484 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.865404+0000", 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.865404+0000", 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.865384+0000", 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999804 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866205+0000", 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082046199999999999 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866220+0000", 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5087e-05 2024-09-16T06:48:49.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866228+0000", 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9389999999999997e-06 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866252+0000", 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4298e-05 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866329+0000", 2024-09-16T06:48:49.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7231999999999995e-05 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.867064+0000", 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000734843 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870412+0000", 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033478000000000002 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.876793+0000", 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063807739999999996 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.876843+0000", 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9965e-05 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:485 3.7s0 3:e6e9cc11:::smithi11731613-37:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,15,11,c,8,5] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:48:49.495 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.866237+0000", 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.53779411399999999, 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011784964, 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 485 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866237+0000", 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866237+0000", 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866236+0000", 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866240+0000", 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1509999999999997e-06 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866248+0000", 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3629999999999996e-06 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866256+0000", 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1340000000000004e-06 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.867970+0000", 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001713886 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.868195+0000", 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022532100000000001 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.869222+0000", 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001026625 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870904+0000", 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001681838 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.877963+0000", 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070592850000000002 2024-09-16T06:48:49.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.878022+0000", 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9168e-05 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:486 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.866267+0000", 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.53776404, 2024-09-16T06:48:49.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013542332000000001, 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 486 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866267+0000", 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866267+0000", 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866267+0000", 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866270+0000", 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.608e-06 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866278+0000", 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6690000000000006e-06 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.866285+0000", 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000004e-06 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870346+0000", 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0040604869999999998 2024-09-16T06:48:49.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870403+0000", 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7192999999999999e-05 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.878313+0000", 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0079104190000000001 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.878418+0000", 2024-09-16T06:48:49.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000104957 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.879809+0000", 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001391055 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.506 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:49.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:48:49.508 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:49.506+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:49.609 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:49.607+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:49.633 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:49.633 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:49.633 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:49.633 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:49.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:48:49.710 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:49.708+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:49.771 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:49.771 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:49.771 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:48:49.771 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:48:49.771 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:49.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:48:49.795 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:49.811 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:49.807+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:49.817 INFO:tasks.rados.rados.0.smithi117.stdout:185: done (3 left) 2024-09-16T06:48:49.818 INFO:tasks.rados.rados.0.smithi117.stdout:186: done (2 left) 2024-09-16T06:48:49.818 INFO:tasks.rados.rados.0.smithi117.stdout:188: done (1 left) 2024-09-16T06:48:49.818 INFO:tasks.rados.rados.0.smithi117.stdout:189: done (0 left) 2024-09-16T06:48:49.818 INFO:tasks.rados.rados.0.smithi117.stdout:190: read oid 31 snap -1 2024-09-16T06:48:49.818 INFO:tasks.rados.rados.0.smithi117.stdout:190: expect (ObjNum 73 snap 7 seq_num 73) 2024-09-16T06:48:49.818 INFO:tasks.rados.rados.0.smithi117.stdout:191: snap_create 2024-09-16T06:48:49.856 INFO:tasks.rados.rados.0.smithi117.stdout:190: expect (ObjNum 73 snap 7 seq_num 73) 2024-09-16T06:48:49.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:48:49.912 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:49.911+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:49.949 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:49.949 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=4, reqid=client.4511.0:40, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.033240558999999, 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055223370000000004, 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918169+0000", 2024-09-16T06:48:49.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918371+0000", 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020196999999999999 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918465+0000", 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4196000000000004e-05 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918494+0000", 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9254000000000001e-05 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918552+0000", 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8031000000000001e-05 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918561+0000", 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3950000000000007e-06 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.923659+0000", 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050976270000000004 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.923691+0000", 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2160999999999999e-05 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 35/32 ECSubWrite(tid=11, reqid=client.4511.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.935919+0000", 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.015490856, 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050956999999999999, 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935919+0000", 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935919+0000", 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935917+0000", 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935936+0000", 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8315e-05 2024-09-16T06:48:49.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935988+0000", 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2194000000000002e-05 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936000+0000", 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2277e-05 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936118+0000", 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117835 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936141+0000", 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3425999999999999e-05 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.940932+0000", 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004790356 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941014+0000", 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2599e-05 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:52 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.015300710999998, 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011921681999999999, 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.959 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 52 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.960 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936107+0000", 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936869+0000", 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076224400000000003 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936883+0000", 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3704e-05 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936893+0000", 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0567000000000001e-05 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937221+0000", 2024-09-16T06:48:49.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032796600000000001 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937632+0000", 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00041103599999999999 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939883+0000", 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022507009999999999 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.963 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943370+0000", 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034869190000000002 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.947880+0000", 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045105839999999998 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.948030+0000", 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015010800000000001 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.964 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:53 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.014499437000001, 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.679887028, 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 53 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.965 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936909+0000", 2024-09-16T06:48:49.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936915+0000", 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8130000000000001e-06 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936921+0000", 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9939999999999997e-06 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936925+0000", 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9920000000000001e-06 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941426+0000", 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045008390000000004 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941657+0000", 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023140500000000001 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.942737+0000", 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001079637 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943730+0000", 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099337199999999996 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616662+0000", 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67293178399999998 2024-09-16T06:48:49.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616797+0000", 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135186 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:54 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.014481202999999, 2024-09-16T06:48:49.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2642334100000001, 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 54 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936930+0000", 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.725e-06 2024-09-16T06:48:49.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936933+0000", 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.698e-06 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936967+0000", 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3618000000000002e-05 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943520+0000", 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065532869999999997 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943693+0000", 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017226899999999999 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617527+0000", 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67383411699999995 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617765+0000", 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023803499999999999 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201162+0000", 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58339697800000001 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.es0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-16T06:48:49.975 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.332431348, 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58220860799999996, 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:48:49.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618977+0000", 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618996+0000", 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9266000000000002e-05 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200456+0000", 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58146017500000002 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200480+0000", 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3186000000000001e-05 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200604+0000", 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012390599999999999 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201187+0000", 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00058315500000000002 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 40/37 ECSubWrite(tid=20, reqid=client.4511.0:219, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.970537865000001, 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005811274, 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.979 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980871+0000", 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981331+0000", 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046021300000000001 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981469+0000", 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138088 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981496+0000", 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7353000000000001e-05 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981653+0000", 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000156816 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981681+0000", 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.73e-05 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986615+0000", 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049340180000000001 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986683+0000", 2024-09-16T06:48:49.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8258999999999998e-05 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 49/37 ECSubWrite(tid=34, reqid=osd.7.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=49'22))", 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:36.861594+0000", 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 13.089815636000001, 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050575910000000002, 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861594+0000", 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861594+0000", 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861593+0000", 2024-09-16T06:48:49.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861602+0000", 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0359999999999995e-06 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861654+0000", 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2295e-05 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861662+0000", 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6000000000000001e-06 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861710+0000", 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7537000000000002e-05 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.861719+0000", 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5070000000000006e-06 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.866627+0000", 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049081480000000002 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.866651+0000", 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4226000000000001e-05 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:433 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~737280] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9990346309999998, 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013805337000000001, 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 433 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952372+0000", 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953751+0000", 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013786060000000001 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953786+0000", 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5605999999999998e-05 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953802+0000", 2024-09-16T06:48:49.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5478999999999999e-05 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953976+0000", 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000173979 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.954152+0000", 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017637499999999999 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.957075+0000", 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029234159999999999 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.960418+0000", 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003342579 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965998+0000", 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055795719999999997 2024-09-16T06:48:49.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966180+0000", 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000182578 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:434 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~573440] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9896546529999997, 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011138353, 2024-09-16T06:48:49.994 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 434 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961752+0000", 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962728+0000", 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00097593799999999998 2024-09-16T06:48:49.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962755+0000", 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6597e-05 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962769+0000", 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3583e-05 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962853+0000", 2024-09-16T06:48:49.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4399000000000003e-05 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.963021+0000", 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000167625 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964326+0000", 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013054519999999999 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:49.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972047+0000", 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007721042 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972718+0000", 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00067065799999999997 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972893+0000", 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000175472 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:49.999 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:435 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~196608] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9864334069999998, 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086272310000000008, 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:50.000 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 435 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964973+0000", 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965356+0000", 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038219399999999998 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965388+0000", 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2276000000000001e-05 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965403+0000", 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966329+0000", 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00092638200000000001 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966612+0000", 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000282525 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.968300+0000", 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016878450000000001 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973229+0000", 2024-09-16T06:48:50.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049290990000000002 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973498+0000", 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026954399999999998 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973603+0000", 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105053 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.005 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:436 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (57) in=7b] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9859939410000003, 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.01058251, 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 436 2024-09-16T06:48:50.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965414+0000", 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965422+0000", 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.43e-06 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965435+0000", 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3332999999999999e-05 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965445+0000", 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4800000000000007e-06 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.969503+0000", 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0040581259999999996 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.969750+0000", 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024736799999999998 2024-09-16T06:48:50.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.970629+0000", 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087915900000000002 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975095+0000", 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044660719999999998 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:48:50.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975865+0000", 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076922299999999998 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975998+0000", 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000133442 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:437 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9859558709999998, 2024-09-16T06:48:50.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014457724, 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 437 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965453+0000", 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965457+0000", 2024-09-16T06:48:50.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0500000000000002e-06 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965471+0000", 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4403000000000001e-05 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965479+0000", 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7859999999999997e-06 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.974299+0000", 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0088199260000000005 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.974438+0000", 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000139519 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.976547+0000", 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021088449999999998 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.976785+0000", 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023817099999999999 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.979911+0000", 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031258570000000001 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.016 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 55/37 ECSubWriteReply(tid=6, last_complete=55'5, committed=1, applied=1))", 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.966619+0000", 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9847905810000004, 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052951489999999999, 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966619+0000", 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966619+0000", 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966617+0000", 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966628+0000", 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.137e-05 2024-09-16T06:48:50.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966692+0000", 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3379000000000004e-05 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966709+0000", 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6670000000000001e-05 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.971849+0000", 2024-09-16T06:48:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051400070000000003 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.971914+0000", 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5469000000000006e-05 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 55/37 ECSubWriteReply(tid=6, last_complete=55'5, committed=1, applied=1))", 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9846636679999996, 2024-09-16T06:48:50.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061836670000000003, 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966745+0000", 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966753+0000", 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9759999999999995e-06 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966796+0000", 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3655999999999997e-05 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966805+0000", 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1200000000000008e-06 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972502+0000", 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056967390000000001 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972929+0000", 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000427253 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 55/37 ECSubWriteReply(tid=6, last_complete=55'5, committed=1, applied=1))", 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.967048+0000", 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.9843614780000003, 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053582480000000004, 2024-09-16T06:48:50.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.967048+0000", 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.967048+0000", 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.967047+0000", 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.967057+0000", 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0736e-05 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.967127+0000", 2024-09-16T06:48:50.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9906000000000005e-05 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.967140+0000", 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2927e-05 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972376+0000", 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052358420000000001 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972406+0000", 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.025e-05 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 61/37 ECSubWrite(tid=39, reqid=osd.4.0:39, at_version=61'34, trim_to=0'0, roll_forward_to=58'32))", 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.1261678420000001, 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059543850000000004, 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825241+0000", 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825303+0000", 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1846000000000004e-05 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825467+0000", 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000164575 2024-09-16T06:48:50.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825495+0000", 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7705999999999999e-05 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.828570+0000", 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030753009999999999 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.828603+0000", 2024-09-16T06:48:50.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2546999999999999e-05 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831167+0000", 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002564588 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831196+0000", 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8532000000000001e-05 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.032 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 61/54 ECSubWrite(tid=121, reqid=osd.3.0:121, at_version=61'51, trim_to=0'0, roll_forward_to=57'47))", 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.1027286570000001, 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063343469999999997, 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:48:50.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848679+0000", 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848734+0000", 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5569999999999998e-05 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848826+0000", 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1281999999999998e-05 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848851+0000", 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4837e-05 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848916+0000", 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5440999999999999e-05 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848926+0000", 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7359999999999996e-06 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854992+0000", 2024-09-16T06:48:50.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060664559999999996 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.855015+0000", 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2955e-05 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 61/37 ECSubWrite(tid=41, reqid=client.4511.0:482, at_version=61'36, trim_to=0'0, roll_forward_to=61'34))", 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.852497+0000", 2024-09-16T06:48:50.037 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.0989124379999999, 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057364180000000001, 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852497+0000", 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852497+0000", 2024-09-16T06:48:50.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852496+0000", 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852592+0000", 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6198000000000003e-05 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852630+0000", 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7988999999999998e-05 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852638+0000", 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2269999999999993e-06 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852698+0000", 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9623000000000002e-05 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.852707+0000", 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2080000000000006e-06 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.858208+0000", 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055006389999999999 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.858233+0000", 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5485000000000001e-05 2024-09-16T06:48:50.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/37 ECSubWriteReply(tid=16, last_complete=61'13, committed=1, applied=1))", 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 1.0965529540000001, 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.246763658, 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:48:50.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854856+0000", 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:48:50.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854862+0000", 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7479999999999999e-06 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101354+0000", 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.24649253400000001 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101371+0000", 2024-09-16T06:48:50.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7116e-05 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101507+0000", 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013554600000000001 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101620+0000", 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011343199999999999 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.045 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.046 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:50.046 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:50.046 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:50.048 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:48:50.048 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:50.048 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:50.049 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:50.049 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:50.050 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:50.043+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:50.066 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:50.088 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:50.089 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:50.113 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:50.113 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:50.137 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:50.137 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:50.148 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:50.146+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:50.160 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:50.161 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:50.165 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:48:50.165 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:48:50.165 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:50.179 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:48:50.180 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:50.181 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:50.203 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:50.203 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:50.226 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:50.227 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:50.248 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:50.248+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:50.249 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:50.249 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:50.270 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:50.270 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:50.288 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:50.288 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:50.293 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:50.293 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:50.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:50.300+0000 7f594b7fe640 0 -- 172.21.15.100:0/1489803765 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7f594c21edb0 msgr2=0x7f594c2211a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:50.301 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:48:50.324 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:50.324 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:50.347 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:50.347 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:50.349 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:50.347+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:50.430 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:50.430 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:50.443 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:48:50.488 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:50.486+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:50.552 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:50.550+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:50.567 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:50.567 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:50.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:48:50.653 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:50.656+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:50.740 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:50.740 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:50.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:48:50.754 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:50.751+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:50.818 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:48:50.820 INFO:tasks.rados.rados.0.smithi117.stdout:190: done (1 left) 2024-09-16T06:48:50.820 INFO:tasks.rados.rados.0.smithi117.stdout:191: done (0 left) 2024-09-16T06:48:50.820 INFO:tasks.rados.rados.0.smithi117.stdout:192: rmattr oid 26 current snap is 19 2024-09-16T06:48:50.820 INFO:tasks.rados.rados.0.smithi117.stdout:192: done (0 left) 2024-09-16T06:48:50.820 INFO:tasks.rados.rados.0.smithi117.stdout:193: rmattr oid 47 current snap is 19 2024-09-16T06:48:50.820 INFO:tasks.rados.rados.0.smithi117.stdout:193: done (0 left) 2024-09-16T06:48:50.821 INFO:tasks.rados.rados.0.smithi117.stdout:194: rmattr oid 22 current snap is 19 2024-09-16T06:48:50.821 INFO:tasks.rados.rados.0.smithi117.stdout:194: done (0 left) 2024-09-16T06:48:50.821 INFO:tasks.rados.rados.0.smithi117.stdout:195: snap_remove snap 2 2024-09-16T06:48:50.854 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:50.851+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:50.871 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:50.871 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:50.885 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:48:50.955 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:50.951+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:51.007 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.008 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.022 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:48:51.055 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:51.056+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:51.142 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.143 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.157 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:51.155+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:51.157 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:48:51.257 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:51.254+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:51.272 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.272 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.286 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:48:51.358 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:51.355+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:51.449 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.450 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.458 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:51.460+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:51.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:48:51.560 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:51.555+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:51.579 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.579 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:48:51.660 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:51.659+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:51.711 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.711 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.725 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:48:51.761 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:51.759+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:51.816 INFO:tasks.rados.rados.0.smithi117.stdout:195: done (0 left) 2024-09-16T06:48:51.816 INFO:tasks.rados.rados.0.smithi117.stdout:196: snap_create 2024-09-16T06:48:51.861 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.861 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:51.862 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:51.859+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:51.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:48:51.963 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:51.963+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:51.986 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:51.986 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:52.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:48:52.063 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:52.058+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:52.122 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:52.137 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:48:52.164 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:52.164+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:52.264 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:52.259+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:52.267 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:52.267 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:52.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:48:52.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:52.366 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:52.364+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:52.401 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:52.401 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:52.401 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:52.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:52.402 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T06:48:52.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T06:48:52.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:48:52.466 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:52.463+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:52.527 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:52.527 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:52.535 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:52.535 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:52.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:48:52.552 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:52.552 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:52.568 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:52.563+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:52.575 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:52.576 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:52.605 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:52.621 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:52.647 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:52.669 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:52.666+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:52.669 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:52.669 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:52.696 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:52.697 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:52.710 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:52.710 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:52.710 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:52.710 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 64, 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "b2976256-637c-4e28-94f6-99f20017bc5e", 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout: "created": "2024-09-16T06:47:53.157733+0000", 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout: "modified": "2024-09-16T06:48:51.808211+0000", 2024-09-16T06:48:52.714 INFO:teuthology.orchestra.run.smithi100.stdout: "last_up_change": "2024-09-16T06:48:06.616578+0000", 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "last_in_change": "2024-09-16T06:48:24.894403+0000", 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_num": 5799936, 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_set": [ 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "pglog_hardlimit", 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "purged_snapdirs", 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_deletes", 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "sortbitwise" 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_version": 6, 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T06:48:52.715 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_snapshot": "", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_max": 3, 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd": 16, 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "require_min_compat_client": "luminous", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "min_compat_client": "jewel", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "require_osd_release": "reef", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_crimson": false, 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "pools": [ 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 1, 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": ".mgr", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:05.838641+0000", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:52.716 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 1, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 1, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 1, 2024-09-16T06:48:52.717 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 1, 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 1, 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "25", 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:52.718 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:52.719 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:52.720 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_max": 32, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 1 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.721 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": {} 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 15.380000114440918, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 15.380000114440918, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.12999999523162842, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 0.97786140441894531, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "rbd", 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:08.489111+0000", 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:52.723 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 8, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 8, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 8, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 8, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 8, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:52.724 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "28", 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 3, 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 28, 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:52.726 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:52.727 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd": {} 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 4.0199999809265137, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 4.0199999809265137, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.56000000238418579, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2.25, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2.25, 2024-09-16T06:48:52.728 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 0.97786140441894531, 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "unique_pool_0", 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:18.885917+0000", 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:48:52.729 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 3, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 7, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 5, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 1, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:48:52.730 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 16, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 16, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 16, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 16, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 16, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "64", 2024-09-16T06:48:52.731 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 30, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 64, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:48:52.732 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "shecprofile", 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:48:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 16384, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: "rados": {} 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.734 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "osds": [ 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "54494294-f25f-41f2-bff7-106492665e07", 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:52.735 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6808", 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6809", 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.736 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6810", 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.737 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6811", 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6814", 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6815", 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6812", 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6813", 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6809/1028275949", 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6811/1028275949", 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6815/1028275949", 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6813/1028275949", 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.740 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "a265e752-1cd6-497e-bcb9-c56591a5ce30", 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.741 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6808", 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6809", 2024-09-16T06:48:52.742 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6811", 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6813", 2024-09-16T06:48:52.743 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6819", 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.744 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6821", 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6815", 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6817", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6809/930331367", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6813/930331367", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6821/930331367", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6817/930331367", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.746 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "dada34c6-4dc6-4f9f-bc31-85983fd2d7fd", 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.747 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6800", 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6801", 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.748 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6802", 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6803", 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.749 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6806", 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6807", 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.750 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6804", 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6805", 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6801/4231506893", 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6803/4231506893", 2024-09-16T06:48:52.751 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6807/4231506893", 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6805/4231506893", 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "6c0fc051-a4a0-4ce5-989a-668e75bbe3c0", 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.752 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 54, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6802", 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6803", 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6804", 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6805", 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6808", 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.755 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6809", 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6806", 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6807", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6803/1995864012", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6805/1995864012", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6809/1995864012", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6807/1995864012", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "36a46387-b859-4937-bbd0-5dd708630c6e", 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:52.758 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.759 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6802", 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6803", 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.760 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6806", 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6807", 2024-09-16T06:48:52.761 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6804", 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6805", 2024-09-16T06:48:52.762 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6801/2054405220", 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6803/2054405220", 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6807/2054405220", 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6805/2054405220", 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:52.763 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "073f3d23-77f0-4fd3-b067-27fa81137288", 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.764 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6810", 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6812", 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.765 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6814", 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6816", 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.766 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6822", 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6823", 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.767 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6818", 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6820", 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6812/1081990569", 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6816/1081990569", 2024-09-16T06:48:52.768 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6823/1081990569", 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6820/1081990569", 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "8a81592f-3704-48f6-89ba-cc4997c2105e", 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.769 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6824", 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6825", 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6826", 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6827", 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6830", 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6831", 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6828", 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6829", 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6825/1095047955", 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6827/1095047955", 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6831/1095047955", 2024-09-16T06:48:52.774 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6829/1095047955", 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27", 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 0.645782470703125, 2024-09-16T06:48:52.775 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6818", 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6819", 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6820", 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6821", 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6824", 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6825", 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6822", 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6823", 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6819/3607859516", 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6821/3607859516", 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6825/3607859516", 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6823/3607859516", 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.780 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7ce38131-057f-4d45-88a5-dae8bcceae3c", 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.781 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6816", 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6817", 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.782 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6818", 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6819", 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6822", 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6823", 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6820", 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6821", 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.785 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6817/232055141", 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6819/232055141", 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6823/232055141", 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6821/232055141", 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "aa7fde6e-6563-4e37-8af2-9114d0091362", 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.786 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6824", 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6825", 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.788 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6826", 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6827", 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.789 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6830", 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6831", 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6828", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6829", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6825/1511881207", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6827/1511881207", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6831/1511881207", 2024-09-16T06:48:52.791 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6829/1511881207", 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "05c5389d-8082-4be3-aa6d-138e00720f06", 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.792 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6808", 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6809", 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6810", 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6811", 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6814", 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6815", 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6812", 2024-09-16T06:48:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6813", 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6809/435900412", 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6811/435900412", 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6815/435900412", 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6813/435900412", 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.797 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "99552590-1d0a-420e-b0c3-357246880658", 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.798 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6810", 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6811", 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.799 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6812", 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6813", 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6816", 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6817", 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.801 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6814", 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6815", 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6811/3208528083", 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6813/3208528083", 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6817/3208528083", 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6815/3208528083", 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "165fea1c-74b6-4806-a0d9-47441e161ed5", 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.803 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6824", 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.805 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.805 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6825", 2024-09-16T06:48:52.805 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.805 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.808 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.808 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.808 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.808 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6826", 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6827", 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.809 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6830", 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6831", 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6828", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6829", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6825/3107910928", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6827/3107910928", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6831/3107910928", 2024-09-16T06:48:52.811 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6829/3107910928", 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "03402d0c-e7fd-4233-b2cd-bf79cd75e9bc", 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 0, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 0, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.812 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6800", 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.813 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6801", 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6802", 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.814 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6803", 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6806", 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6807", 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6804", 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6805", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6801/316232366", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6803/316232366", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6807/316232366", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6805/316232366", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.817 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7b027faf-5c9e-4aee-9cb9-9c6459b8fe39", 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:48:52.818 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6816", 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6817", 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.819 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6818", 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6819", 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.820 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6822", 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6823", 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.821 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6820", 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6821", 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:48:52.822 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6817/693497656", 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6819/693497656", 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6823/693497656", 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6821/693497656", 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.823 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "4877aa40-c0d8-494f-8a98-36089ed694c2", 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:48:52.824 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6826", 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6827", 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.825 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6828", 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6829", 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.826 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6832", 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6833", 2024-09-16T06:48:52.827 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6830", 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6831", 2024-09-16T06:48:52.828 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6827/579912865", 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6829/579912865", 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6833/579912865", 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6831/579912865", 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.829 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_xinfo": [ 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.584963+0000", 2024-09-16T06:48:52.830 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.677023+0000", 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.831 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.608639+0000", 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.832 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.384489+0000", 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.480551+0000", 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.476690+0000", 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:48:52.834 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.615096+0000", 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.835 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.581224+0000", 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.597357+0000", 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.549502+0000", 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:48:52.837 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.448485+0000", 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.838 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.495825+0000", 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.628407+0000", 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 65536, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.368452+0000", 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.600045+0000", 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:48:52.841 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.567594+0000", 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap": [], 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_items": [], 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_primaries": [], 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_temp": [], 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_temp": [], 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklist": {}, 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "range_blocklist": {}, 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profiles": { 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "default": { 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "2", 2024-09-16T06:48:52.842 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "1", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "jerasure", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "reed_sol_van" 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "shecprofile": { 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "c": "2", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-device-class": "", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-root": "default", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "4", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "3", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "shecprofile", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "shec", 2024-09-16T06:48:52.843 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "multiple" 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps_queue": [ 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 5, 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 13, 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:52.844 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: "new_removed_snaps": [ 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 5, 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.845 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "new_purged_snaps": [ 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 11, 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 14, 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:48:52.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 19, 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_node_flags": {}, 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "device_class_flags": {}, 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode": { 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_enabled": false, 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_bucket_count": 0, 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "degraded_stretch_mode": 0, 2024-09-16T06:48:52.847 INFO:teuthology.orchestra.run.smithi100.stdout: "recovering_stretch_mode": 0, 2024-09-16T06:48:52.848 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_bucket": 0 2024-09-16T06:48:52.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:48:52.848 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:48:52.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:48:52.850 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:52.850 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:52.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:48:52.852 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:52.847+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:52.853 INFO:tasks.thrashosds.thrasher:j is {'epoch': 64, 'fsid': 'b2976256-637c-4e28-94f6-99f20017bc5e', 'created': '2024-09-16T06:47:53.157733+0000', 'modified': '2024-09-16T06:48:51.808211+0000', 'last_up_change': '2024-09-16T06:48:06.616578+0000', 'last_in_change': '2024-09-16T06:48:24.894403+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T06:48:05.838641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9778614044189453, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T06:48:08.489111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9778614044189453, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-16T06:48:18.885917+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '54494294-f25f-41f2-bff7-106492665e07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6808', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6809', 'nonce': 1028275949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6810', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6811', 'nonce': 1028275949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6814', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6815', 'nonce': 1028275949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6812', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6813', 'nonce': 1028275949}]}, 'public_addr': '172.21.15.100:6809/1028275949', 'cluster_addr': '172.21.15.100:6811/1028275949', 'heartbeat_back_addr': '172.21.15.100:6815/1028275949', 'heartbeat_front_addr': '172.21.15.100:6813/1028275949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a265e752-1cd6-497e-bcb9-c56591a5ce30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 930331367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6811', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 930331367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6819', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 930331367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6815', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 930331367}]}, 'public_addr': '172.21.15.107:6809/930331367', 'cluster_addr': '172.21.15.107:6813/930331367', 'heartbeat_back_addr': '172.21.15.107:6821/930331367', 'heartbeat_front_addr': '172.21.15.107:6817/930331367', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'dada34c6-4dc6-4f9f-bc31-85983fd2d7fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 4231506893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 4231506893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 4231506893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 4231506893}]}, 'public_addr': '172.21.15.115:6801/4231506893', 'cluster_addr': '172.21.15.115:6803/4231506893', 'heartbeat_back_addr': '172.21.15.115:6807/4231506893', 'heartbeat_front_addr': '172.21.15.115:6805/4231506893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6c0fc051-a4a0-4ce5-989a-668e75bbe3c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6802', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6803', 'nonce': 1995864012}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6804', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6805', 'nonce': 1995864012}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6808', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6809', 'nonce': 1995864012}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6806', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6807', 'nonce': 1995864012}]}, 'public_addr': '172.21.15.117:6803/1995864012', 'cluster_addr': '172.21.15.117:6805/1995864012', 'heartbeat_back_addr': '172.21.15.117:6809/1995864012', 'heartbeat_front_addr': '172.21.15.117:6807/1995864012', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '36a46387-b859-4937-bbd0-5dd708630c6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 2054405220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6802', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6803', 'nonce': 2054405220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6806', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6807', 'nonce': 2054405220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6804', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6805', 'nonce': 2054405220}]}, 'public_addr': '172.21.15.100:6801/2054405220', 'cluster_addr': '172.21.15.100:6803/2054405220', 'heartbeat_back_addr': '172.21.15.100:6807/2054405220', 'heartbeat_front_addr': '172.21.15.100:6805/2054405220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '073f3d23-77f0-4fd3-b067-27fa81137288', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6812', 'nonce': 1081990569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6816', 'nonce': 1081990569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 1081990569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6820', 'nonce': 1081990569}]}, 'public_addr': '172.21.15.107:6812/1081990569', 'cluster_addr': '172.21.15.107:6816/1081990569', 'heartbeat_back_addr': '172.21.15.107:6823/1081990569', 'heartbeat_front_addr': '172.21.15.107:6820/1081990569', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8a81592f-3704-48f6-89ba-cc4997c2105e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 1095047955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1095047955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1095047955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1095047955}]}, 'public_addr': '172.21.15.115:6825/1095047955', 'cluster_addr': '172.21.15.115:6827/1095047955', 'heartbeat_back_addr': '172.21.15.115:6831/1095047955', 'heartbeat_front_addr': '172.21.15.115:6829/1095047955', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.645782470703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6818', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6819', 'nonce': 3607859516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6820', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6821', 'nonce': 3607859516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6824', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6825', 'nonce': 3607859516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6822', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6823', 'nonce': 3607859516}]}, 'public_addr': '172.21.15.117:6819/3607859516', 'cluster_addr': '172.21.15.117:6821/3607859516', 'heartbeat_back_addr': '172.21.15.117:6825/3607859516', 'heartbeat_front_addr': '172.21.15.117:6823/3607859516', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '7ce38131-057f-4d45-88a5-dae8bcceae3c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6816', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6817', 'nonce': 232055141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6818', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6819', 'nonce': 232055141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6822', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6823', 'nonce': 232055141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6820', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6821', 'nonce': 232055141}]}, 'public_addr': '172.21.15.100:6817/232055141', 'cluster_addr': '172.21.15.100:6819/232055141', 'heartbeat_back_addr': '172.21.15.100:6823/232055141', 'heartbeat_front_addr': '172.21.15.100:6821/232055141', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'aa7fde6e-6563-4e37-8af2-9114d0091362', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 1511881207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 1511881207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 1511881207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 1511881207}]}, 'public_addr': '172.21.15.107:6825/1511881207', 'cluster_addr': '172.21.15.107:6827/1511881207', 'heartbeat_back_addr': '172.21.15.107:6831/1511881207', 'heartbeat_front_addr': '172.21.15.107:6829/1511881207', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '05c5389d-8082-4be3-aa6d-138e00720f06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 435900412}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 435900412}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 435900412}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 435900412}]}, 'public_addr': '172.21.15.115:6809/435900412', 'cluster_addr': '172.21.15.115:6811/435900412', 'heartbeat_back_addr': '172.21.15.115:6815/435900412', 'heartbeat_front_addr': '172.21.15.115:6813/435900412', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '99552590-1d0a-420e-b0c3-357246880658', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6810', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6811', 'nonce': 3208528083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6812', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6813', 'nonce': 3208528083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6816', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6817', 'nonce': 3208528083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6814', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6815', 'nonce': 3208528083}]}, 'public_addr': '172.21.15.117:6811/3208528083', 'cluster_addr': '172.21.15.117:6813/3208528083', 'heartbeat_back_addr': '172.21.15.117:6817/3208528083', 'heartbeat_front_addr': '172.21.15.117:6815/3208528083', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '165fea1c-74b6-4806-a0d9-47441e161ed5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6824', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6825', 'nonce': 3107910928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6826', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6827', 'nonce': 3107910928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6830', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6831', 'nonce': 3107910928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6828', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6829', 'nonce': 3107910928}]}, 'public_addr': '172.21.15.100:6825/3107910928', 'cluster_addr': '172.21.15.100:6827/3107910928', 'heartbeat_back_addr': '172.21.15.100:6831/3107910928', 'heartbeat_front_addr': '172.21.15.100:6829/3107910928', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '03402d0c-e7fd-4233-b2cd-bf79cd75e9bc', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 316232366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 316232366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 316232366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 316232366}]}, 'public_addr': '172.21.15.107:6801/316232366', 'cluster_addr': '172.21.15.107:6803/316232366', 'heartbeat_back_addr': '172.21.15.107:6807/316232366', 'heartbeat_front_addr': '172.21.15.107:6805/316232366', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '7b027faf-5c9e-4aee-9cb9-9c6459b8fe39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 693497656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 693497656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 693497656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 693497656}]}, 'public_addr': '172.21.15.115:6817/693497656', 'cluster_addr': '172.21.15.115:6819/693497656', 'heartbeat_back_addr': '172.21.15.115:6823/693497656', 'heartbeat_front_addr': '172.21.15.115:6821/693497656', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '4877aa40-c0d8-494f-8a98-36089ed694c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6826', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6827', 'nonce': 579912865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6828', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6829', 'nonce': 579912865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 579912865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6830', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6831', 'nonce': 579912865}]}, 'public_addr': '172.21.15.117:6827/579912865', 'cluster_addr': '172.21.15.117:6829/579912865', 'heartbeat_back_addr': '172.21.15.117:6833/579912865', 'heartbeat_front_addr': '172.21.15.117:6831/579912865', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.584963+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.677023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.608639+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.384489+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.480551+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.476690+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.615096+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.581224+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.597357+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.549502+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.448485+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.495825+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.628407+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.368452+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.600045+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.567594+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 13, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 5, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 11, 'length': 1}, {'begin': 14, 'length': 1}, {'begin': 19, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T06:48:52.853 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T06:48:52.854 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:196: done (0 left) 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:197: copy_from oid 15 from oid 2 current snap is 20 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:198: copy_from oid 12 from oid 7 current snap is 20 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:199: copy_from oid 13 from oid 9 current snap is 20 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 21 current snap is 20 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:200: seq_num 96 ranges {0=458752,458752=688128,1146880=475136} 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:200: writing smithi11731613-21 from 0 to 458752 tid 1 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:200: writing smithi11731613-21 from 458752 to 1146880 tid 2 2024-09-16T06:48:52.854 INFO:tasks.rados.rados.0.smithi117.stdout:200: writing smithi11731613-21 from 1146880 to 1622016 tid 3 2024-09-16T06:48:52.855 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 49 current snap is 20 2024-09-16T06:48:52.855 INFO:tasks.rados.rados.0.smithi117.stdout:201: seq_num 97 ranges {819200=475136,1294336=442368,1736704=442368,2179072=278528} 2024-09-16T06:48:52.855 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:52.855 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:52.855 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:52.855 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:52.856 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:52.856 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:52.856 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:52.870 INFO:tasks.rados.rados.0.smithi117.stdout:201: writing smithi11731613-49 from 819200 to 1294336 tid 1 2024-09-16T06:48:52.877 INFO:tasks.rados.rados.0.smithi117.stdout:201: writing smithi11731613-49 from 1294336 to 1736704 tid 2 2024-09-16T06:48:52.882 INFO:tasks.rados.rados.0.smithi117.stdout:201: writing smithi11731613-49 from 1736704 to 2179072 tid 3 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:201: writing smithi11731613-49 from 2179072 to 2457600 tid 4 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:202: snap_create 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:199: finishing copy_from to smithi11731613-13 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:199: got expected ENOENT (src dne) 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:199: finishing copy_from racing read to smithi11731613-13 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:198: finishing copy_from racing read to smithi11731613-12 2024-09-16T06:48:52.886 INFO:tasks.rados.rados.0.smithi117.stdout:198: finishing copy_from to smithi11731613-12 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 12 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:197: finishing copy_from racing read to smithi11731613-15 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:200: finishing write tid 1 to smithi11731613-21 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:197: finishing copy_from to smithi11731613-15 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 15 v 68 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:200: finishing write tid 2 to smithi11731613-21 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:200: finishing write tid 3 to smithi11731613-21 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:200: finishing write tid 4 to smithi11731613-21 2024-09-16T06:48:52.887 INFO:tasks.rados.rados.0.smithi117.stdout:200: finishing write tid 5 to smithi11731613-21 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:200: oid 21 updating version 0 to 78 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:200: oid 21 updating version 78 to 80 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:200: oid 21 version 80 is already newer than 79 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:200: oid 21 version 80 is already newer than 77 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 21 v 80 (ObjNum 96 snap 20 seq_num 96) dirty exists 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:200: left oid 21 (ObjNum 96 snap 20 seq_num 96) 2024-09-16T06:48:52.888 INFO:tasks.rados.rados.0.smithi117.stdout:201: finishing write tid 1 to smithi11731613-49 2024-09-16T06:48:52.897 INFO:tasks.rados.rados.0.smithi117.stdout:201: finishing write tid 2 to smithi11731613-49 2024-09-16T06:48:52.898 INFO:tasks.rados.rados.0.smithi117.stdout:201: finishing write tid 3 to smithi11731613-49 2024-09-16T06:48:52.899 INFO:tasks.rados.rados.0.smithi117.stdout:201: finishing write tid 4 to smithi11731613-49 2024-09-16T06:48:52.901 INFO:tasks.rados.rados.0.smithi117.stdout:201: finishing write tid 5 to smithi11731613-49 2024-09-16T06:48:52.950 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:52.948+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:53.023 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:53.024 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:53.038 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:48:53.052 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:53.050+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:53.067 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:53.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:53.072+0000 7f1091b0a640 0 -- 172.21.15.100:0/1659636861 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7f108c21ec00 msgr2=0x7f108c220ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:53.110 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:53.111 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:53.152 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:53.152 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:53.153 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:53.147+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:53.165 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:53.165 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:53.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:53.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.511955043, 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:48:53.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:48:53.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:48:53.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.880302907000001, 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017661530000000002, 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-09-16T06:48:53.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:53.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282229+0000", 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282228+0000", 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282240+0000", 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2339e-05 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282259+0000", 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8499e-05 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282274+0000", 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5673000000000001e-05 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297469+0000", 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015194177999999999 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297683+0000", 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021437800000000001 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.297867+0000", 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018445199999999999 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299716+0000", 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018485590000000001 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299769+0000", 2024-09-16T06:48:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3192999999999997e-05 2024-09-16T06:48:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299836+0000", 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7255000000000006e-05 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299891+0000", 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4243999999999998e-05 2024-09-16T06:48:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.880242226999997, 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018410340000000001, 2024-09-16T06:48:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-16T06:48:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282290+0000", 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282289+0000", 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282302+0000", 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3049e-05 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282319+0000", 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7379000000000001e-05 2024-09-16T06:48:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.282336+0000", 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6881e-05 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.298971+0000", 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016634903999999999 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299117+0000", 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014632600000000001 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.299317+0000", 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019987600000000001 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300036+0000", 2024-09-16T06:48:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071873499999999997 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300588+0000", 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055181800000000004 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300655+0000", 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7742000000000003e-05 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.300700+0000", 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999998e-05 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.698154864999999, 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:48:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:48:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.698026313, 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:48:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:48:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:48:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.697988985999999, 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:48:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:48:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:48:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:48:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.697853987999999, 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:48:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:48:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:48:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.697821826999999, 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:48:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:48:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.697793396000002, 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:48:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:48:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:48:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.697686476999998, 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:48:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:48:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.246168498000003, 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018030338, 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 165 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916364+0000", 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916363+0000", 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916367+0000", 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4790000000000001e-06 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916371+0000", 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3460000000000004e-06 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.916374+0000", 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4400000000000001e-06 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924029+0000", 2024-09-16T06:48:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076542229999999999 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924097+0000", 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8849000000000004e-05 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.924193+0000", 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5544999999999994e-05 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.927143+0000", 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029495250000000002 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934287+0000", 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071441270000000001 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934346+0000", 2024-09-16T06:48:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9521000000000002e-05 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.934394+0000", 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-05 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.004179966000002, 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:48:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:48:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.004149718000001, 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:48:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:48:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.004120282999999, 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.004009363999998, 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:48:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:48:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:48:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:48:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:77 3.8s0 3:119b2380:::smithi11731613-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.190839837999999, 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055956466000000003, 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 77 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:48:53.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971691+0000", 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971701+0000", 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8519999999999999e-06 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971715+0000", 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4386000000000001e-05 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971725+0000", 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0360000000000001e-05 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.974948+0000", 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003222759 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975027+0000", 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8497000000000004e-05 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975520+0000", 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049320500000000003 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.977378+0000", 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018576230000000001 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027491+0000", 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050112973999999998 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027649+0000", 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015816800000000001 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.273 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:78 3.8s0 3:119b2380:::smithi11731613-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.190744823999999, 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059293289999999998, 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:48:53.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971786+0000", 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971793+0000", 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971809+0000", 2024-09-16T06:48:53.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5225000000000001e-05 2024-09-16T06:48:53.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971820+0000", 2024-09-16T06:48:53.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0922e-05 2024-09-16T06:48:53.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975891+0000", 2024-09-16T06:48:53.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040709759999999996 2024-09-16T06:48:53.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:53.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.976067+0000", 2024-09-16T06:48:53.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017645900000000001 2024-09-16T06:48:53.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027711+0000", 2024-09-16T06:48:53.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051643840000000003 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027855+0000", 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014441399999999999 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031081+0000", 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032252359999999998 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 35/32 ECSubWriteReply(tid=162, last_complete=35'4, committed=1, applied=1))", 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.129779361000001, 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48350030599999999, 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032752+0000", 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032763+0000", 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1807000000000001e-05 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516055+0000", 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48329140799999998 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516068+0000", 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2782999999999999e-05 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516198+0000", 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129949 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516253+0000", 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5390000000000003e-05 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=17, reqid=client.4511.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.106683097000001, 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.51123769299999999, 2024-09-16T06:48:53.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055848+0000", 2024-09-16T06:48:53.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055863+0000", 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.464e-05 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565781+0000", 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50991880999999994 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565796+0000", 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4263e-05 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565932+0000", 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013609900000000001 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565949+0000", 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7229000000000001e-05 2024-09-16T06:48:53.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567019+0000", 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001070463 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567087+0000", 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7229000000000003e-05 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.7s2 40/37 ECSubRead(tid=11, to_read={3:e60b6e1b:::smithi11731613-14:head=0,4096,0}, subchunks={3:e60b6e1b:::smithi11731613-14:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.095001702000001, 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.375282063, 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067529+0000", 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067547+0000", 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8187000000000001e-05 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442125+0000", 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.37457846099999997 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442142+0000", 2024-09-16T06:48:53.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6843000000000001e-05 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442313+0000", 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017038200000000001 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442812+0000", 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049983999999999998 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.296 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:53.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:48:53.298 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:53.295+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:53.398 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:53.395+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:53.400 INFO:tasks.rados.rados.0.smithi117.stdout:201: finishing write tid 6 to smithi11731613-49 2024-09-16T06:48:53.400 INFO:tasks.rados.rados.0.smithi117.stdout:201: oid 49 updating version 0 to 47 2024-09-16T06:48:53.401 INFO:tasks.rados.rados.0.smithi117.stdout:201: oid 49 updating version 47 to 49 2024-09-16T06:48:53.401 INFO:tasks.rados.rados.0.smithi117.stdout:201: oid 49 version 49 is already newer than 45 2024-09-16T06:48:53.401 INFO:tasks.rados.rados.0.smithi117.stdout:201: oid 49 version 49 is already newer than 48 2024-09-16T06:48:53.401 INFO:tasks.rados.rados.0.smithi117.stdout:201: oid 49 version 49 is already newer than 46 2024-09-16T06:48:53.401 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 49 v 49 (ObjNum 97 snap 20 seq_num 97) dirty exists 2024-09-16T06:48:53.401 INFO:tasks.rados.rados.0.smithi117.stdout:201: left oid 49 (ObjNum 97 snap 20 seq_num 97) 2024-09-16T06:48:53.426 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:53.427 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:53.427 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:53.427 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:53.440 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:48:53.528 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:53.527+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:53.600 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:53.598+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:53.605 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:53.605 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:53.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:48:53.701 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:53.702+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:53.794 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.894727435, 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022146409999999998, 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897264+0000", 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897309+0000", 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5123000000000001e-05 2024-09-16T06:48:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913004+0000", 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015694784999999999 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913046+0000", 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2945999999999998e-05 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914353+0000", 2024-09-16T06:48:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306104 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914377+0000", 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4562999999999998e-05 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919353+0000", 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049759360000000002 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919412+0000", 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9327000000000003e-05 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:22 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.892451892, 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023053033000000001, 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 22 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899530+0000", 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999881 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901021+0000", 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014909019999999999 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901056+0000", 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5136999999999999e-05 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901077+0000", 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1622999999999999e-05 2024-09-16T06:48:53.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901274+0000", 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019618500000000001 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901606+0000", 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033268099999999998 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903794+0000", 2024-09-16T06:48:53.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021876339999999999 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907586+0000", 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003792489 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922327+0000", 2024-09-16T06:48:53.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014741124 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922595+0000", 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026710500000000001 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:23 3.3s0 3:ce1274e9:::smithi11731613-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.890888690000001, 2024-09-16T06:48:53.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022719541999999999, 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901103+0000", 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901115+0000", 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1722999999999999e-05 2024-09-16T06:48:53.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901134+0000", 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9213000000000001e-05 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901149+0000", 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.523e-05 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905252+0000", 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041033170000000004 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905520+0000", 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026748400000000002 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.906744+0000", 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012242209999999999 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923321+0000", 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016576822000000001 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923623+0000", 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249399999999999 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923824+0000", 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000200955 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:24 3.3s0 3:ce1274e9:::smithi11731613-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.890829864000001, 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033126111999999999, 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-16T06:48:53.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:48:53.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901162+0000", 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901169+0000", 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901187+0000", 2024-09-16T06:48:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8320000000000001e-05 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901203+0000", 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5872999999999999e-05 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907294+0000", 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006090949 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907512+0000", 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021778499999999999 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924007+0000", 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016495668000000002 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924218+0000", 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021115199999999999 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934290+0000", 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010071302000000001 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:31 3.3s0 3:c7654c4c:::smithi11731613-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.890777540999999, 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026931531000000002, 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901215+0000", 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902241+0000", 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010258190000000001 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902275+0000", 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4037e-05 2024-09-16T06:48:53.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902312+0000", 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.718e-05 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:48:53.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053663829999999997 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.908030+0000", 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035200800000000001 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912004+0000", 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039736770000000001 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923909+0000", 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011905429 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927953+0000", 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040441499999999998 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928147+0000", 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194005 2024-09-16T06:48:53.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:32 3.3s0 3:c7654c4c:::smithi11731613-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.889661483000001, 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028234555000000001, 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 32 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902330+0000", 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902341+0000", 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1279e-05 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902360+0000", 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902374+0000", 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3917000000000001e-05 2024-09-16T06:48:53.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917230+0000", 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014855607 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917486+0000", 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025648799999999997 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918644+0000", 2024-09-16T06:48:53.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011572749999999999 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928363+0000", 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097195389999999993 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930283+0000", 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919505 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930566+0000", 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028369099999999998 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:33 3.3s0 3:c7654c4c:::smithi11731613-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.889606792999999, 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037765197, 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 33 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902386+0000", 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902392+0000", 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893999999999998e-05 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902429+0000", 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8689999999999999e-05 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934639+0000", 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032210404999999998 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934843+0000", 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020376199999999999 2024-09-16T06:48:53.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.940152+0000", 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053090200000000002 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs2 34/32 ECSubWrite(tid=3, reqid=client.4511.0:17, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.903724+0000", 2024-09-16T06:48:53.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.888268983, 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019548528999999999, 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903724+0000", 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903724+0000", 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903722+0000", 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903764+0000", 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1103999999999998e-05 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904106+0000", 2024-09-16T06:48:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034275700000000001 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904129+0000", 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2819999999999998e-05 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904806+0000", 2024-09-16T06:48:53.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067673099999999997 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.904829+0000", 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.321e-05 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922886+0000", 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018056961 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923273+0000", 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038695 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=4, reqid=client.4511.0:44, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.874690106999999, 2024-09-16T06:48:53.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788766000000001, 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917302+0000", 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917349+0000", 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7796000000000001e-05 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941504+0000", 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024154637 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941525+0000", 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0608999999999999e-05 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941640+0000", 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011531000000000001 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941662+0000", 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2328999999999999e-05 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943041+0000", 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001379047 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943092+0000", 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0834000000000002e-05 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:68 3.3s0 3:cbbd057c:::smithi11731613-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.837968492999998, 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045964098000000002, 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 68 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954024+0000", 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954031+0000", 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954046+0000", 2024-09-16T06:48:53.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954058+0000", 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1938e-05 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956544+0000", 2024-09-16T06:48:53.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024857949999999998 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956742+0000", 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019819 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.957650+0000", 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090843 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959725+0000", 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074256 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999914+0000", 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040189895000000003 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999989+0000", 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4527e-05 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:69 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.837924008000002, 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048288992000000003, 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 69 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954068+0000", 2024-09-16T06:48:53.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954074+0000", 2024-09-16T06:48:53.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954088+0000", 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4866e-05 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954100+0000", 2024-09-16T06:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1328999999999999e-05 2024-09-16T06:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959103+0000", 2024-09-16T06:48:53.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035749999999997 2024-09-16T06:48:53.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:53.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959283+0000", 2024-09-16T06:48:53.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017953899999999999 2024-09-16T06:48:53.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000081+0000", 2024-09-16T06:48:53.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040797893000000002 2024-09-16T06:48:53.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000261+0000", 2024-09-16T06:48:53.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180304 2024-09-16T06:48:53.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002358+0000", 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002097392 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:98 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:48:53.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.796500141999999, 2024-09-16T06:48:53.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.086611296000000004, 2024-09-16T06:48:53.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:48:53.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:48:53.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995492+0000", 2024-09-16T06:48:53.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995501+0000", 2024-09-16T06:48:53.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8720000000000008e-06 2024-09-16T06:48:53.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995517+0000", 2024-09-16T06:48:53.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6650000000000002e-05 2024-09-16T06:48:53.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995529+0000", 2024-09-16T06:48:53.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1447999999999999e-05 2024-09-16T06:48:53.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.997874+0000", 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023456250000000001 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998037+0000", 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016212600000000001 2024-09-16T06:48:53.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998796+0000", 2024-09-16T06:48:53.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075897099999999999 2024-09-16T06:48:53.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002377+0000", 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00358195 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081972+0000", 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.079594867 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082105+0000", 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013220899999999999 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:99 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.796433079, 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.087893389000000002, 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-16T06:48:53.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995559+0000", 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995565+0000", 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2659999999999998e-06 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995581+0000", 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.611e-05 2024-09-16T06:48:53.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995594+0000", 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2269000000000001e-05 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999116+0000", 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003522238 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:53.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999227+0000", 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111483 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082346+0000", 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.083118743999999994 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082416+0000", 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9993000000000006e-05 2024-09-16T06:48:53.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.083454+0000", 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00103751 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:158 3.3s0 3:c7654c4c:::smithi11731613-11:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.712207269, 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41982487699999999, 2024-09-16T06:48:53.877 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:48:53.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079784+0000", 2024-09-16T06:48:53.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:53.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079794+0000", 2024-09-16T06:48:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6260000000000007e-06 2024-09-16T06:48:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079810+0000", 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6631000000000001e-05 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079821+0000", 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0849000000000001e-05 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079917+0000", 2024-09-16T06:48:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5730999999999995e-05 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.080126+0000", 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020895399999999999 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081069+0000", 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094364700000000004 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082604+0000", 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001534863 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499525+0000", 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41692029400000002 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499611+0000", 2024-09-16T06:48:53.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-05 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:226 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.836326467999999, 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027980046000000001, 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 226 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:48:53.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955665+0000", 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955672+0000", 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000002e-06 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955690+0000", 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8179999999999999e-05 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955704+0000", 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.396e-05 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956020+0000", 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000316101 2024-09-16T06:48:53.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956038+0000", 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8425999999999999e-05 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974108+0000", 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018069776999999999 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974338+0000", 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022928799999999999 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975586+0000", 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012484639999999999 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.979590+0000", 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040043939999999997 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983481+0000", 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038905480000000002 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983647+0000", 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166045 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:348 3.3s0 3:c742dae1:::smithi11731613-33:head [append 0~622592] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:53.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.951304111999999, 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020152403999999999, 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 348 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840687+0000", 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841715+0000", 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010285260000000001 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841743+0000", 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7990000000000001e-05 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841758+0000", 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4686e-05 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842034+0000", 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027588800000000001 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842198+0000", 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000164599 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.843642+0000", 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014432760000000001 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.847983+0000", 2024-09-16T06:48:53.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043409349999999998 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860716+0000", 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012732902000000001 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860842+0000", 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012617499999999999 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:350 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (57) in=7b] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.950069244000002, 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019634848, 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 350 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841923+0000", 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841929+0000", 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6060000000000002e-06 2024-09-16T06:48:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841938+0000", 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4970000000000007e-06 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841946+0000", 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9130000000000003e-06 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.848163+0000", 2024-09-16T06:48:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062167339999999998 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.848319+0000", 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015559300000000001 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849250+0000", 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093117299999999998 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.853284+0000", 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040342399999999997 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.861427+0000", 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081432659999999997 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.861559+0000", 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013169599999999999 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:351 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.950040088000002, 2024-09-16T06:48:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023956603, 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 351 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841957+0000", 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0199999999999996e-06 2024-09-16T06:48:53.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841966+0000", 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-06 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841974+0000", 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.34e-06 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849799+0000", 2024-09-16T06:48:53.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078258089999999995 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.850139+0000", 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033906799999999999 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862313+0000", 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012174318999999999 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862630+0000", 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031676099999999998 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.865910+0000", 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032802859999999999 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 47/32 ECSubWrite(tid=68, reqid=client.4511.0:361, at_version=0'0, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.920475996, 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50083384399999997, 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871516+0000", 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871541+0000", 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5782000000000002e-05 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370610+0000", 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.49906899100000002 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370625+0000", 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.429e-05 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370744+0000", 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119746 2024-09-16T06:48:53.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370773+0000", 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8807e-05 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372302+0000", 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015286519999999999 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372351+0000", 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9348999999999998e-05 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 49/32 ECSubWrite(tid=70, reqid=osd.3.0:70, at_version=49'51, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.936788824000001, 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023397583, 2024-09-16T06:48:53.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855204+0000", 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967296 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855221+0000", 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877682+0000", 2024-09-16T06:48:53.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022460635999999999 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877690+0000", 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5369999999999997e-06 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877755+0000", 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4584000000000004e-05 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:53.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877766+0000", 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0968000000000001e-05 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878577+0000", 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000811029 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:53.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878602+0000", 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5191e-05 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:53.920 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:197: done (5 left) 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:198: done (4 left) 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:199: done (3 left) 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:200: done (2 left) 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:201: done (1 left) 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:202: done (0 left) 2024-09-16T06:48:53.922 INFO:tasks.rados.rados.0.smithi117.stdout:203: snap_create 2024-09-16T06:48:53.923 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:48:53.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:48:53.924 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:53.919+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:54.023 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:54.023+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:54.098 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:54.098 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:54.098 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:48:54.098 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:54.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:48:54.123 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:54.119+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:54.223 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:54.219+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:54.231 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:48:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:48:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:48:54.231 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:54.246 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:48:54.336 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:54.335+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:54.415 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:48:54.415 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:48:54.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:4 3.2s0 3:42d6f0b3:::smithi11731613-2:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892106+0000", 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.521143893000001, 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013862718, 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4 2024-09-16T06:48:54.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892106+0000", 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892106+0000", 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892103+0000", 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892265+0000", 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000161636 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892292+0000", 2024-09-16T06:48:54.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6716e-05 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892321+0000", 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9516e-05 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892606+0000", 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028500200000000002 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892925+0000", 2024-09-16T06:48:54.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031936399999999999 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.894225+0000", 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012999140000000001 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896826+0000", 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026006570000000001 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905794+0000", 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089676909999999999 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905968+0000", 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000174649 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:5 3.2s0 3:42d6f0b3:::smithi11731613-2:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:54.421 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.520874242000001, 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029483957000000002, 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:48:54.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892373+0000", 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892385+0000", 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1939999999999999e-05 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892403+0000", 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8202999999999998e-05 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892421+0000", 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8189000000000002e-05 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895208+0000", 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786527 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895392+0000", 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000184255 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896349+0000", 2024-09-16T06:48:54.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00095644999999999997 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902998+0000", 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066495039999999997 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921651+0000", 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018652716 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921859+0000", 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208359 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:6 3.2s0 3:42d6f0b3:::smithi11731613-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.520793062999999, 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034217268000000002, 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892455+0000", 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892464+0000", 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.67e-06 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892484+0000", 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0265999999999999e-05 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892500+0000", 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.596e-05 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897287+0000", 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047868049999999999 2024-09-16T06:48:54.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897527+0000", 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023998800000000001 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922181+0000", 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024653615 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922389+0000", 2024-09-16T06:48:54.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020808600000000001 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926674+0000", 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042850639999999999 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:43 3.2s0 3:4636348c:::smithi11731613-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:48:54.432 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.50757501, 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020346564000000001, 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:48:54.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905671+0000", 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909244+0000", 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035731700000000001 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909296+0000", 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2352999999999998e-05 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909321+0000", 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4813000000000001e-05 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909751+0000", 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042994099999999998 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.910155+0000", 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040433999999999999 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912510+0000", 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002354929 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924742+0000", 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012231967 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.925839+0000", 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001096302 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926021+0000", 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000182492 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.437 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:44 3.2s0 3:4636348c:::smithi11731613-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.50388938, 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034540078000000002, 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-16T06:48:54.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909357+0000", 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909370+0000", 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2583e-05 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909390+0000", 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9930000000000001e-05 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909421+0000", 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1285999999999998e-05 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913550+0000", 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041288399999999999 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913768+0000", 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000217691 2024-09-16T06:48:54.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914791+0000", 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010230770000000001 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924746+0000", 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099553599999999999 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943736+0000", 2024-09-16T06:48:54.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018990000999999999 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943900+0000", 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016405 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:45 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.503814444, 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040625472000000003, 2024-09-16T06:48:54.443 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909434+0000", 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909441+0000", 2024-09-16T06:48:54.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0330000000000003e-06 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909460+0000", 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8674000000000001e-05 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909473+0000", 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3485999999999999e-05 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919590+0000", 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010117482000000001 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919812+0000", 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000221389 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944680+0000", 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024868369000000001 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944880+0000", 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019949900000000001 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.950061+0000", 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051807770000000001 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 35/32 ECSubWrite(tid=4, reqid=client.4511.0:40, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.915548+0000", 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.497701403000001, 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012162571, 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915548+0000", 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915548+0000", 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915546+0000", 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915901+0000", 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035479100000000003 2024-09-16T06:48:54.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916173+0000", 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027175799999999998 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916190+0000", 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7054e-05 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916302+0000", 2024-09-16T06:48:54.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112642 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916324+0000", 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1673000000000001e-05 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927667+0000", 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011342703000000001 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927711+0000", 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3930000000000001e-05 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.bs6 35/32 ECSubWrite(tid=5, reqid=client.4511.0:37, at_version=35'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.915913+0000", 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.497336055000002, 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011672927, 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.453 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915913+0000", 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915913+0000", 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.915912+0000", 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916061+0000", 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014851600000000001 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916234+0000", 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000173468 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916249+0000", 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4884e-05 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916436+0000", 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018702200000000001 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.916459+0000", 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2413e-05 2024-09-16T06:48:54.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927521+0000", 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011061807999999999 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927586+0000", 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5833000000000005e-05 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:81 3.2s0 3:476f74d7:::smithi11731613-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.969996+0000", 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.443253401, 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011260137, 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 81 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.458 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.969996+0000", 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.969996+0000", 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.969995+0000", 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.970000+0000", 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.69e-06 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.970008+0000", 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.452e-06 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.970014+0000", 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.088e-06 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.973521+0000", 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003507212 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.973627+0000", 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010581099999999999 2024-09-16T06:48:54.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.979248+0000", 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056215450000000004 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.979423+0000", 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017510200000000001 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.981256+0000", 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001832867 2024-09-16T06:48:54.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:117 3.2s0 3:47b7cdf1:::smithi11731613-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.018679+0000", 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.394570069, 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012405204, 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018679+0000", 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018679+0000", 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018678+0000", 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018684+0000", 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8039999999999998e-06 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018700+0000", 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6130999999999999e-05 2024-09-16T06:48:54.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.018713+0000", 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.238e-05 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.022854+0000", 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041413459999999997 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.022975+0000", 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012055199999999999 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.029496+0000", 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006520896 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.029605+0000", 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109688 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031085+0000", 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001479403 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:230 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.457653343, 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0201987, 2024-09-16T06:48:54.468 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 230 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955593+0000", 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955613+0000", 2024-09-16T06:48:54.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0630000000000001e-05 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955662+0000", 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8702999999999998e-05 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955689+0000", 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.73e-05 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955849+0000", 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015934200000000001 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955874+0000", 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.548e-05 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966673+0000", 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010799329999999999 2024-09-16T06:48:54.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966857+0000", 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018410900000000001 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975795+0000", 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089374099999999998 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:231 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.457526833999999, 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021272859000000002, 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 231 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955721+0000", 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:54.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955733+0000", 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2129e-05 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955755+0000", 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1423000000000001e-05 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955770+0000", 2024-09-16T06:48:54.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4790000000000001e-05 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955974+0000", 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020403199999999999 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955982+0000", 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6940000000000001e-06 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.967793+0000", 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011811056 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.968023+0000", 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023004500000000001 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976996+0000", 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089721929999999998 2024-09-16T06:48:54.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:232 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.457302289000001, 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022137836000000001, 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.479 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 232 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955946+0000", 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955958+0000", 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-05 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955977+0000", 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.912e-05 2024-09-16T06:48:54.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955993+0000", 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5894000000000001e-05 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956101+0000", 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107402 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956123+0000", 2024-09-16T06:48:54.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2056999999999999e-05 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971700+0000", 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015577155000000001 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971931+0000", 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023091400000000001 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.978085+0000", 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061543179999999998 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:298 3.2s0 3:47b7cdf1:::smithi11731613-39:head [append 0~802816] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.127256+0000", 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.285993007999998, 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012207569999999999, 2024-09-16T06:48:54.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 298 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.127256+0000", 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.127256+0000", 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.127254+0000", 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128653+0000", 2024-09-16T06:48:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013990299999999999 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128685+0000", 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1721999999999999e-05 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128701+0000", 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5787e-05 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128802+0000", 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100853 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.128962+0000", 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016078199999999999 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.130997+0000", 2024-09-16T06:48:54.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020347849999999999 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134383+0000", 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003385435 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.139275+0000", 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048926799999999999 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.139464+0000", 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000188822 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:301 3.2s0 3:47b7cdf1:::smithi11731613-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.134619+0000", 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.278630748000001, 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011711747, 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 301 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134619+0000", 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134619+0000", 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134617+0000", 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134625+0000", 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6699999999999994e-06 2024-09-16T06:48:54.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134644+0000", 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.134663+0000", 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9059000000000002e-05 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.140923+0000", 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062592910000000002 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.141178+0000", 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025534400000000001 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.144207+0000", 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030284499999999998 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.144334+0000", 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000127179 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.146331+0000", 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019967909999999999 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:322 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:30.255575+0000", 2024-09-16T06:48:54.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.157674744000001, 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012092308, 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 322 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255575+0000", 2024-09-16T06:48:54.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255575+0000", 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255574+0000", 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255577+0000", 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255583+0000", 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.686e-06 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.255589+0000", 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7400000000000001e-06 2024-09-16T06:48:54.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.260880+0000", 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052912239999999998 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.260988+0000", 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000108108 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.266365+0000", 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053770550000000004 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.266465+0000", 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9570999999999999e-05 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:30.267667+0000", 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001202079 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.500 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:376 3.2s0 3:4636348c:::smithi11731613-15:head [rollback d] snapc e=[e,d,c,b,9,8,6,5] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:36.920165+0000", 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.493084325000002, 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011822129000000001, 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.501 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 376 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920165+0000", 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920165+0000", 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920164+0000", 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920174+0000", 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0366999999999999e-05 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920195+0000", 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0795999999999999e-05 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920210+0000", 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5432999999999999e-05 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.921329+0000", 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001118092 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.921495+0000", 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166816 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.922786+0000", 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012908660000000001 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.924799+0000", 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020130790000000001 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.931938+0000", 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071385420000000003 2024-09-16T06:48:54.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.931987+0000", 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9455e-05 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:377 3.2s0 3:4636348c:::smithi11731613-15:head [append 0~0] snapc e=[e,d,c,b,9,8,6,5] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:36.920226+0000", 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.493023278999999, 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011885725999999999, 2024-09-16T06:48:54.506 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 377 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920226+0000", 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920226+0000", 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920225+0000", 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920231+0000", 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0870000000000003e-06 2024-09-16T06:48:54.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920248+0000", 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7212999999999999e-05 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.920265+0000", 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6773e-05 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.923678+0000", 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034128079999999998 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.923791+0000", 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113243 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.924265+0000", 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047389699999999998 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.924936+0000", 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067128800000000003 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.932059+0000", 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071230829999999997 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.932112+0000", 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2500999999999998e-05 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 55/37 ECSubWrite(tid=121, reqid=client.4511.0:431, at_version=55'56, trim_to=0'0, roll_forward_to=52'54))", 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.595687196, 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.102329752, 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817619+0000", 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7778999999999997e-05 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917448+0000", 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.099828728000000005 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917466+0000", 2024-09-16T06:48:54.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7907e-05 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917613+0000", 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014708200000000001 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917643+0000", 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0315999999999999e-05 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919830+0000", 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021864940000000002 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919892+0000", 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2076000000000007e-05 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 56/37 ECSubWrite(tid=132, reqid=osd.11.0:132, at_version=56'60, trim_to=0'0, roll_forward_to=55'56))", 2024-09-16T06:48:54.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:43.889886+0000", 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.523363639999999, 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012751574999999999, 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.889886+0000", 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:48:54.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.889886+0000", 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.889884+0000", 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.889945+0000", 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0516999999999997e-05 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.890046+0000", 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101296 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.890068+0000", 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1843000000000001e-05 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.892119+0000", 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020506249999999999 2024-09-16T06:48:54.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.892144+0000", 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4885999999999999e-05 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.902524+0000", 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010380033 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:48:54.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:43.902637+0000", 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113899 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:48:54.521 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:48:54.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:48:54.523 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:54.519+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:54.624 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:54.624+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:54.656 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:54.656 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:54.656 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:54.656 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:54.669 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:48:54.724 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:54.723+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:54.787 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:54.788 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:54.788 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:54.788 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:54.788 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:54.801 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:48:54.845 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:54.843+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:54.889 INFO:tasks.rados.rados.0.smithi117.stdout:203: done (0 left) 2024-09-16T06:48:54.889 INFO:tasks.rados.rados.0.smithi117.stdout:204: read oid 18 snap -1 2024-09-16T06:48:54.889 INFO:tasks.rados.rados.0.smithi117.stdout:204: expect (ObjNum 89 snap 14 seq_num 89) 2024-09-16T06:48:54.889 INFO:tasks.rados.rados.0.smithi117.stdout:205: snap_create 2024-09-16T06:48:54.921 INFO:tasks.rados.rados.0.smithi117.stdout:204: expect (ObjNum 89 snap 14 seq_num 89) 2024-09-16T06:48:54.926 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:48:54.923+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:48:54.934 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:54.934 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:54.934 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds1 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.033536718999997, 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018662743999999998, 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:48:54.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898050+0000", 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898094+0000", 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3769000000000001e-05 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915572+0000", 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017478235000000002 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915584+0000", 2024-09-16T06:48:54.937 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1996e-05 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915648+0000", 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3551000000000002e-05 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915661+0000", 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2897e-05 2024-09-16T06:48:54.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916697+0000", 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001036653 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916715+0000", 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7376000000000001e-05 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.939 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:30 3.as0 3:5d93488c:::smithi11731613-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.025017278, 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017066839, 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.940 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 30 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.941 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906571+0000", 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906575+0000", 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1219999999999997e-06 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906585+0000", 2024-09-16T06:48:54.942 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0142e-05 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.906593+0000", 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5340000000000002e-06 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911387+0000", 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0047939760000000001 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.943 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.911456+0000", 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9554000000000005e-05 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918704+0000", 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072483469999999996 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.918831+0000", 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012617499999999999 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.923638+0000", 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048076789999999996 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds1 35/32 ECSubWrite(tid=5, reqid=client.4511.0:50, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.002769911000001, 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023812890999999999, 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.945 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928818+0000", 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928835+0000", 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7011999999999999e-05 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949938+0000", 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.02110269 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.947 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949947+0000", 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0010000000000007e-06 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950024+0000", 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6520000000000006e-05 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950032+0000", 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6750000000000008e-06 2024-09-16T06:48:54.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952592+0000", 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002559296 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952632+0000", 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0243000000000001e-05 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:74 3.as0 3:5d679340:::smithi11731613-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.969312497000001, 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026692272999999999, 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 74 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.950 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962275+0000", 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962280+0000", 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1300000000000003e-06 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962286+0000", 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4529999999999999e-06 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962292+0000", 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5849999999999999e-06 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964248+0000", 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001956667 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964344+0000", 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6174000000000004e-05 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964765+0000", 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042082000000000003 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965862+0000", 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0010968289999999999 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988875+0000", 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023013051 2024-09-16T06:48:54.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988969+0000", 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3350000000000006e-05 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:75 3.as0 3:5d679340:::smithi11731613-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.969284184999999, 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033077179999999998, 2024-09-16T06:48:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 75 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962304+0000", 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962308+0000", 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0600000000000001e-06 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962314+0000", 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.088e-06 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962319+0000", 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1120000000000004e-06 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964978+0000", 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026582540000000001 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965065+0000", 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7108999999999996e-05 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989270+0000", 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024205503 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989359+0000", 2024-09-16T06:48:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9204999999999997e-05 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995382+0000", 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060223200000000003 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=11, last_complete=35'6, committed=1, applied=1))", 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:48:54.960 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.965548526999999, 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022946498999999999, 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:48:54.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966039+0000", 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966044+0000", 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6959999999999996e-06 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.962 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988553+0000", 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022508866999999998 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988563+0000", 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0373e-05 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988804+0000", 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000240289 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988987+0000", 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018304199999999999 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:95 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.939399608999999, 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042547695000000003, 2024-09-16T06:48:54.964 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 95 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.965 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992188+0000", 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992194+0000", 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8760000000000002e-06 2024-09-16T06:48:54.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992204+0000", 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.011e-05 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992212+0000", 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1890000000000001e-06 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993832+0000", 2024-09-16T06:48:54.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001620565 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993886+0000", 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4021999999999999e-05 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994477+0000", 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059046499999999998 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995906+0000", 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014291919999999999 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034552+0000", 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038646106999999999 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034737+0000", 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018489900000000001 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.969 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:96 3.as0 3:5bc02ad9:::smithi11731613-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.939370625999999, 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.058410493000000001, 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.970 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 96 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992221+0000", 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2899999999999998e-06 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992229+0000", 2024-09-16T06:48:54.972 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3080000000000008e-06 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992236+0000", 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.528e-06 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994995+0000", 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027588700000000001 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995064+0000", 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9758000000000005e-05 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035940+0000", 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040876066000000003 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036021+0000", 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0243999999999999e-05 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050629+0000", 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014608009999999999 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=18, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.936288877999999, 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040520444000000003, 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995299+0000", 2024-09-16T06:48:54.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995304+0000", 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5680000000000001e-06 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034371+0000", 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039067118999999997 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034380+0000", 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7949999999999996e-06 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034448+0000", 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8244000000000006e-05 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035820+0000", 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001372306 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.978 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:144 3.as0 3:50fa89c4:::smithi11731613-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.867902408999999, 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38453364499999998, 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 144 2024-09-16T06:48:54.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063691+0000", 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8879999999999996e-06 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063699+0000", 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5799999999999992e-06 2024-09-16T06:48:54.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063705+0000", 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.246e-06 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065212+0000", 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015062820000000001 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065312+0000", 2024-09-16T06:48:54.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010052600000000001 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065706+0000", 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039373300000000001 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070934+0000", 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052282070000000003 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:54.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448115+0000", 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37718074400000001 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448220+0000", 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105104 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:145 3.as0 3:50fa89c4:::smithi11731613-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:54.984 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.86785944, 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38866241499999998, 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 145 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:54.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063732+0000", 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1999999999999999e-06 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063741+0000", 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4700000000000008e-06 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063748+0000", 2024-09-16T06:48:54.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4740000000000002e-06 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066083+0000", 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023351140000000001 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066155+0000", 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1650999999999996e-05 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448700+0000", 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38254526700000002 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448854+0000", 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154306 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.452392+0000", 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035374640000000001 2024-09-16T06:48:54.989 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=23, last_complete=35'13, committed=1, applied=1))", 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 31.86059423, 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37764362699999998, 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.990 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070994+0000", 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:54.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.071000+0000", 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8980000000000001e-06 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447985+0000", 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37698559799999998 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447996+0000", 2024-09-16T06:48:54.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0672999999999999e-05 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448044+0000", 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8338999999999999e-05 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448638+0000", 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059397499999999995 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:302 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 2] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.796726082999999, 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21732169600000001, 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 302 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.994 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134859+0000", 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:54.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134878+0000", 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9264999999999999e-05 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134922+0000", 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3260000000000003e-05 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134944+0000", 2024-09-16T06:48:54.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2603e-05 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135181+0000", 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023672400000000001 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135385+0000", 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020434300000000001 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.347993+0000", 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21260742799999999 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.351735+0000", 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037421830000000001 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352007+0000", 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027182200000000001 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352184+0000", 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017777400000000001 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:303 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 24.796619913000001, 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21552337099999999, 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:54.999 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 303 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134967+0000", 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134976+0000", 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3860000000000007e-06 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134996+0000", 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0234999999999999e-05 2024-09-16T06:48:55.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135015+0000", 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8949000000000001e-05 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135785+0000", 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000770388 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:55.002 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135925+0000", 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013959700000000001 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350066+0000", 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21414113400000001 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350271+0000", 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020513000000000001 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.003 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350492+0000", 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022106200000000001 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:360 3.as0 3:5d93488c:::smithi11731613-10:head [rollback c] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.857527+0000", 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.074061286999999, 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014127305, 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.004 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 360 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857527+0000", 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857527+0000", 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857526+0000", 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857535+0000", 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0089999999999996e-06 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.006 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857555+0000", 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0329000000000001e-05 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857567+0000", 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1883e-05 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.860280+0000", 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027131630000000002 2024-09-16T06:48:55.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.860451+0000", 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017102999999999999 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.861742+0000", 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012901500000000001 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.865767+0000", 2024-09-16T06:48:55.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0040252650000000001 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.871507+0000", 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0057397280000000004 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.871655+0000", 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014817399999999999 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:361 3.as0 3:5d93488c:::smithi11731613-10:head [append 0~0] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.857620+0000", 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.073968379, 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014241820000000001, 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 361 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.010 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857620+0000", 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857620+0000", 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857619+0000", 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857631+0000", 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2089999999999999e-05 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857652+0000", 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0715000000000001e-05 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.857669+0000", 2024-09-16T06:48:55.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7458000000000001e-05 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.862510+0000", 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048402530000000001 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.862789+0000", 2024-09-16T06:48:55.013 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027985 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.864684+0000", 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001894165 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.867925+0000", 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0032413989999999998 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.871791+0000", 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0038656070000000001 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.871862+0000", 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1552999999999997e-05 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s2 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:48:55.015 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.836681+0000", 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.094907767999999, 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.020203395999999998, 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836681+0000", 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836681+0000", 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836680+0000", 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836717+0000", 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6936e-05 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.017 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836965+0000", 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00024771 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836975+0000", 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0896999999999999e-05 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.839828+0000", 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002852135 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.839853+0000", 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5069e-05 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856839+0000", 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016986543 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856884+0000", 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5213999999999998e-05 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.019 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:368 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (57) in=7b] snapc e=[e,d,c,b,9,8,6,5] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.891488+0000", 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.040100926000001, 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025167017999999999, 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.020 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 368 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891488+0000", 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891488+0000", 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891486+0000", 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891496+0000", 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0237e-05 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891516+0000", 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0307999999999999e-05 2024-09-16T06:48:55.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891531+0000", 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4513e-05 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.894517+0000", 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0029859410000000002 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.894851+0000", 2024-09-16T06:48:55.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0003346 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.896033+0000", 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001181489 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.898299+0000", 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022659519999999999 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.916437+0000", 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018137820999999998 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.916655+0000", 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021813400000000001 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.fs5 61/37 ECSubWrite(tid=159, reqid=osd.11.0:159, at_version=61'68, trim_to=0'0, roll_forward_to=57'64))", 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.828828+0000", 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 6.102760366, 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018879105, 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828828+0000", 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.026 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828828+0000", 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828826+0000", 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828884+0000", 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8081000000000002e-05 2024-09-16T06:48:55.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828997+0000", 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112429 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.829020+0000", 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3366e-05 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.830864+0000", 2024-09-16T06:48:55.028 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0018436489999999999 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.830882+0000", 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8626000000000001e-05 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.847640+0000", 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016757398 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.847708+0000", 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7853000000000002e-05 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 61/32 ECSubWriteReply(tid=126, last_complete=61'99, committed=1, applied=1))", 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 6.0559062749999999, 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63424799499999995, 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.030 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875681+0000", 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875692+0000", 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1154e-05 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508662+0000", 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63297013099999999 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508675+0000", 2024-09-16T06:48:55.032 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3217000000000001e-05 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508759+0000", 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3652999999999993e-05 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.509930+0000", 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001171486 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.033 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.034 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.034 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.034 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:55.035 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:48:55.036 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:55.035+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:55.136 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:55.138+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:55.204 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:55.205 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:55.205 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:55.205 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:55.218 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:48:55.236 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:55.235+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:55.298 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:48:55.337 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:55.338+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:55.340 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:55.341 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:55.341 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:48:55.341 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:48:55.341 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:55.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:48:55.438 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:55.435+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:55.465 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:55.466 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:1 3.ds0 3:b273fc7d:::smithi11731613-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:55.474 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.578881946999999, 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011215246, 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 1 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.475 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892973+0000", 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.476 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893050+0000", 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6996999999999995e-05 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893075+0000", 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5406999999999999e-05 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.477 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893092+0000", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6541999999999998e-05 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893183+0000", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1404999999999996e-05 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893334+0000", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015069399999999999 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894090+0000", 2024-09-16T06:48:55.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00075619499999999998 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901851+0000", 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0077611449999999997 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012789380000000001 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.479 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904191+0000", 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001060636 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.578747982000003, 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:48:55.480 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.481 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.482 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.484 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.485 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:55.486 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.578664185999997, 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.487 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.488 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.489 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.490 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.490 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:48:55.490 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:48:55.490 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.490 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.491 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.492 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.547258947000003, 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.493 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.494 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.495 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.496 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:48:55.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.498 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.547209178999999, 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.499 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.501 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.502 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:48:55.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.439471513999997, 2024-09-16T06:48:55.504 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:48:55.505 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.507 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:48:55.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.509 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:48:55.510 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.438916055, 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:48:55.513 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.514 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:48:55.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:55.517 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.438859723999997, 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:55.518 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:48:55.519 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.520 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:55.521 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:48:55.522 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:219 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [copy-from ver 4] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:48:55.523 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.515640573999999, 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031057211000000001, 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 219 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:48:55.524 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956214+0000", 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.525 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956234+0000", 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0275000000000002e-05 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956284+0000", 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0219999999999997e-05 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.526 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956309+0000", 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4930999999999999e-05 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956442+0000", 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013270999999999999 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956459+0000", 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7317999999999999e-05 2024-09-16T06:48:55.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960744+0000", 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0042850090000000002 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961009+0000", 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026485300000000001 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.528 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.976584+0000", 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015575011999999999 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.981658+0000", 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050738470000000003 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987101+0000", 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054435580000000003 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.529 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987274+0000", 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017305700000000001 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:220 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.515518115999999, 2024-09-16T06:48:55.530 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023931746, 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 220 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.531 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956338+0000", 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.532 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956351+0000", 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2849e-05 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956375+0000", 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4195000000000001e-05 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.533 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956391+0000", 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5352000000000001e-05 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956650+0000", 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025920099999999997 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.534 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956669+0000", 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9157999999999999e-05 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961818+0000", 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051493709999999998 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:55.535 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961948+0000", 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012956 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979661+0000", 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017713421 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.536 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.537 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.537 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979887+0000", 2024-09-16T06:48:55.537 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022550399999999999 2024-09-16T06:48:55.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.537 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.980271+0000", 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038454100000000002 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.538 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 40/32 ECSubWrite(tid=28, reqid=client.4511.0:226, at_version=0'0, trim_to=0'0, roll_forward_to=40'19))", 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.985513+0000", 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.486344750000001, 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0097361930000000006, 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985513+0000", 2024-09-16T06:48:55.539 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985513+0000", 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985511+0000", 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:55.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985541+0000", 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9677999999999999e-05 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985646+0000", 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105235 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.541 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.985671+0000", 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5335000000000001e-05 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.986750+0000", 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001078608 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.542 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.986775+0000", 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5327999999999999e-05 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995186+0000", 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084108599999999992 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.543 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995249+0000", 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3151999999999994e-05 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.544 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:316 3.ds0 3:b2c197ab:::smithi11731613-17:head [copy-from ver 4] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.309842613000001, 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017020408000000001, 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 316 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.545 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162012+0000", 2024-09-16T06:48:55.546 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162030+0000", 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8688000000000001e-05 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.547 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162067+0000", 2024-09-16T06:48:55.548 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6526000000000002e-05 2024-09-16T06:48:55.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.548 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.548 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.548 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162083+0000", 2024-09-16T06:48:55.548 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6061000000000001e-05 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162190+0000", 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107351 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162511+0000", 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032124399999999999 2024-09-16T06:48:55.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.174275+0000", 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011763691999999999 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.176204+0000", 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019286329999999999 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:48:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.178962+0000", 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027583099999999999 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.179036+0000", 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3456999999999998e-05 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:317 3.ds0 3:b2c197ab:::smithi11731613-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:55.551 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.309757277999999, 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013682241, 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 317 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:55.552 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162106+0000", 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4620000000000001e-06 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162124+0000", 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7706999999999998e-05 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.554 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162136+0000", 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2285e-05 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162962+0000", 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000826123 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163038+0000", 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5371999999999996e-05 2024-09-16T06:48:55.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175269+0000", 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01223169 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175524+0000", 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025445300000000003 2024-09-16T06:48:55.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.557 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.557 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.557 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175783+0000", 2024-09-16T06:48:55.557 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025890899999999998 2024-09-16T06:48:55.557 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.558 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.558 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.558 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.558 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:352 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:55.558 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842404+0000", 2024-09-16T06:48:55.559 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.629453878, 2024-09-16T06:48:55.559 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01003983, 2024-09-16T06:48:55.559 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.559 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.559 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 352 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842404+0000", 2024-09-16T06:48:55.560 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.561 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.561 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.561 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842404+0000", 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842400+0000", 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842421+0000", 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0271999999999998e-05 2024-09-16T06:48:55.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842447+0000", 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6692000000000001e-05 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842462+0000", 2024-09-16T06:48:55.563 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4676e-05 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842520+0000", 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7707000000000002e-05 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842621+0000", 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000100756 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.564 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.852444+0000", 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0098231069999999993 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:353 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.629375649, 2024-09-16T06:48:55.565 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011292597, 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 353 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842481+0000", 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842490+0000", 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4760000000000004e-06 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842509+0000", 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8924999999999999e-05 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842523+0000", 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4446e-05 2024-09-16T06:48:55.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.843021+0000", 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000497589 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.843096+0000", 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5332000000000003e-05 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853775+0000", 2024-09-16T06:48:55.569 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010678309 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:354 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.629319720000002, 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013614067000000001, 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:55.570 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 354 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.571 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842537+0000", 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842545+0000", 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2500000000000006e-06 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842562+0000", 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7512000000000001e-05 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842573+0000", 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0976e-05 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844555+0000", 2024-09-16T06:48:55.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001981919 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844621+0000", 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5792999999999998e-05 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.856152+0000", 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011530883 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/37 ECSubReadReply(tid=31, attrs_read=0))", 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.846380+0000", 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.625477823000001, 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0074597700000000001, 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.846380+0000", 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.846380+0000", 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.846379+0000", 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847105+0000", 2024-09-16T06:48:55.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00072596300000000004 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850412+0000", 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0033067159999999999 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850422+0000", 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0081999999999999e-05 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.577 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.852556+0000", 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021343970000000001 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853840+0000", 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001283392 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/37 ECSubReadReply(tid=32, attrs_read=0))", 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.847119+0000", 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.624738894, 2024-09-16T06:48:55.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0068832110000000002, 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847119+0000", 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847119+0000", 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847118+0000", 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.847640+0000", 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00052163399999999999 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850445+0000", 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0028049139999999999 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.850454+0000", 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3950000000000007e-06 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853899+0000", 2024-09-16T06:48:55.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034450779999999999 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.854002+0000", 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010305 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 49/32 ECSubWrite(tid=55, reqid=osd.10.0:55, at_version=49'38, trim_to=0'0, roll_forward_to=47'34))", 2024-09-16T06:48:55.582 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.836348+0000", 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.63551, 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0068904989999999996, 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836348+0000", 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836348+0000", 2024-09-16T06:48:55.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836346+0000", 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836406+0000", 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0553000000000001e-05 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836491+0000", 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5208000000000001e-05 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836517+0000", 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6018999999999999e-05 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.585 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836632+0000", 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011497000000000001 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836655+0000", 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2795000000000001e-05 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.843169+0000", 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065135510000000002 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.843238+0000", 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9472000000000002e-05 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 61/32 ECSubWrite(tid=85, reqid=osd.10.0:85, at_version=61'70, trim_to=0'0, roll_forward_to=59'66))", 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.838586+0000", 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 6.6332717389999996, 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070765639999999996, 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:48:55.587 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838586+0000", 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838586+0000", 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838584+0000", 2024-09-16T06:48:55.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838747+0000", 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016296600000000001 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838885+0000", 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013838500000000001 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838905+0000", 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9850000000000001e-05 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839021+0000", 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000115389 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839036+0000", 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5505000000000001e-05 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.845588+0000", 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065517320000000002 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.845663+0000", 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4697000000000007e-05 2024-09-16T06:48:55.591 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.592 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.592 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.592 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:48:55.592 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:48:55.592 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:55.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:55.593 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:55.594 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:55.594 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:55.595 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:48:55.595 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:55.595 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:55.596 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:55.596 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:55.596 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:55.592+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:55.601 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:55.601 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:55.631 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:55.631 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:55.659 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:55.660 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:55.687 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:55.687 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:55.694 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:55.692+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:55.716 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:55.717 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:55.748 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:55.748 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:55.770 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:48:55.770 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:48:55.770 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:48:55.770 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:48:55.778 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:55.779 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:55.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:48:55.794 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:55.792+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:55.810 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:55.843 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:55.843 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:55.876 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:48:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:55.876+0000 7fe3e2d9d640 0 -- 172.21.15.100:0/580801322 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7fe3e4223a20 msgr2=0x7fe3e4225e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-16T06:48:55.895 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:55.896+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:55.897 INFO:tasks.rados.rados.0.smithi117.stdout:204: done (1 left) 2024-09-16T06:48:55.897 INFO:tasks.rados.rados.0.smithi117.stdout:205: done (0 left) 2024-09-16T06:48:55.897 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 28 current snap is 23 2024-09-16T06:48:55.897 INFO:tasks.rados.rados.0.smithi117.stdout:206: seq_num 98 ranges {458752=491520,950272=589824,1540096=557056,2097152=16384} 2024-09-16T06:48:55.905 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:55.905 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:55.912 INFO:tasks.rados.rados.0.smithi117.stdout:206: writing smithi11731613-28 from 458752 to 950272 tid 1 2024-09-16T06:48:55.921 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:48:55.923 INFO:tasks.rados.rados.0.smithi117.stdout:206: writing smithi11731613-28 from 950272 to 1540096 tid 2 2024-09-16T06:48:55.925 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:48:55.932 INFO:tasks.rados.rados.0.smithi117.stdout:206: writing smithi11731613-28 from 1540096 to 2097152 tid 3 2024-09-16T06:48:55.932 INFO:tasks.rados.rados.0.smithi117.stdout:206: writing smithi11731613-28 from 2097152 to 2113536 tid 4 2024-09-16T06:48:55.933 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 3 current snap is 23 2024-09-16T06:48:55.933 INFO:tasks.rados.rados.0.smithi117.stdout:206: finishing write tid 1 to smithi11731613-28 2024-09-16T06:48:55.933 INFO:tasks.rados.rados.0.smithi117.stdout:206: finishing write tid 2 to smithi11731613-28 2024-09-16T06:48:55.933 INFO:tasks.rados.rados.0.smithi117.stdout:207: seq_num 99 ranges {1163264=507904,1671168=606208,2277376=557056} 2024-09-16T06:48:55.959 INFO:tasks.rados.rados.0.smithi117.stdout:207: writing smithi11731613-3 from 1163264 to 1671168 tid 1 2024-09-16T06:48:55.968 INFO:tasks.rados.rados.0.smithi117.stdout:207: writing smithi11731613-3 from 1671168 to 2277376 tid 2 2024-09-16T06:48:55.975 INFO:tasks.rados.rados.0.smithi117.stdout:207: writing smithi11731613-3 from 2277376 to 2834432 tid 3 2024-09-16T06:48:55.975 INFO:tasks.rados.rados.0.smithi117.stdout:208: snap_remove snap 16 2024-09-16T06:48:55.997 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:55.995+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:56.045 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.045 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.060 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:48:56.097 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:56.096+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:56.101 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:56.101 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:56.198 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:56.196+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.240 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:48:56.299 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:56.299+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:56.356 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.356 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:48:56.417 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:56.415+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:56.488 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.488 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.502 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:56.499+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:56.502 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:48:56.603 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:56.599+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:56.633 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.633 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:48:56.704 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:48:56.704+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:48:56.771 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.771 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:48:56.805 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:56.803+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:56.813 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:208: done (2 left) 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:209: rollback oid 30 current snap is 23 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 30 to 19 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: finishing write tid 3 to smithi11731613-28 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: finishing write tid 4 to smithi11731613-28 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: finishing write tid 5 to smithi11731613-28 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: finishing write tid 6 to smithi11731613-28 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: oid 28 updating version 0 to 71 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: oid 28 version 71 is already newer than 70 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: oid 28 version 71 is already newer than 69 2024-09-16T06:48:56.815 INFO:tasks.rados.rados.0.smithi117.stdout:206: oid 28 version 71 is already newer than 68 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:206: oid 28 version 71 is already newer than 67 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 28 v 71 (ObjNum 98 snap 23 seq_num 98) dirty exists 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:206: left oid 28 (ObjNum 98 snap 23 seq_num 98) 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: finishing write tid 1 to smithi11731613-3 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: finishing write tid 2 to smithi11731613-3 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: finishing write tid 3 to smithi11731613-3 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: finishing write tid 4 to smithi11731613-3 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: finishing write tid 5 to smithi11731613-3 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: oid 3 updating version 0 to 27 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: oid 3 updating version 27 to 28 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: oid 3 updating version 28 to 30 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:207: oid 3 version 30 is already newer than 29 2024-09-16T06:48:56.816 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 30 (ObjNum 99 snap 23 seq_num 99) dirty exists 2024-09-16T06:48:56.817 INFO:tasks.rados.rados.0.smithi117.stdout:207: left oid 3 (ObjNum 99 snap 23 seq_num 99) 2024-09-16T06:48:56.817 INFO:tasks.rados.rados.0.smithi117.stdout:206: done (2 left) 2024-09-16T06:48:56.817 INFO:tasks.rados.rados.0.smithi117.stdout:207: done (1 left) 2024-09-16T06:48:56.817 INFO:tasks.rados.rados.0.smithi117.stdout:210: rollback oid 44 current snap is 23 2024-09-16T06:48:56.817 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 44 to 11 2024-09-16T06:48:56.817 INFO:tasks.rados.rados.0.smithi117.stdout:211: snap_create 2024-09-16T06:48:56.838 INFO:tasks.rados.rados.0.smithi117.stdout:210: finishing rollback tid 0 to smithi11731613-44 2024-09-16T06:48:56.840 INFO:tasks.rados.rados.0.smithi117.stdout:210: finishing rollback tid 1 to smithi11731613-44 2024-09-16T06:48:56.843 INFO:tasks.rados.rados.0.smithi117.stdout:209: finishing rollback tid 0 to smithi11731613-30 2024-09-16T06:48:56.851 INFO:tasks.rados.rados.0.smithi117.stdout:209: finishing rollback tid 1 to smithi11731613-30 2024-09-16T06:48:56.853 INFO:tasks.rados.rados.0.smithi117.stdout:209: finishing rollback tid 2 to smithi11731613-30 2024-09-16T06:48:56.853 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 113 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-16T06:48:56.906 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:48:56.906+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:48:56.939 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:56.939 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:56.954 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:48:57.007 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:57.003+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:57.077 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.077 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:48:57.107 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:57.103+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:57.209 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:57.207+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:57.215 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.216 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:48:57.309 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:57.310+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:57.349 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.349 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:48:57.411 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:48:57.407+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:48:57.481 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.481 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.495 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:48:57.497 INFO:tasks.rados.rados.0.smithi117.stdout:210: finishing rollback tid 2 to smithi11731613-44 2024-09-16T06:48:57.497 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 78 (ObjNum 63 snap 4 seq_num 63) dirty exists 2024-09-16T06:48:57.511 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:57.510+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:57.611 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.612 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.612 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:57.614+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:57.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:48:57.713 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:57.714+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:57.747 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.748 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.762 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:48:57.814 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:57.811+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:57.817 INFO:tasks.rados.rados.0.smithi117.stdout:209: done (2 left) 2024-09-16T06:48:57.817 INFO:tasks.rados.rados.0.smithi117.stdout:210: done (1 left) 2024-09-16T06:48:57.817 INFO:tasks.rados.rados.0.smithi117.stdout:211: done (0 left) 2024-09-16T06:48:57.817 INFO:tasks.rados.rados.0.smithi117.stdout:212: snap_remove snap 5 2024-09-16T06:48:57.855 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:48:57.855 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:48:57.855 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-16T06:48:57.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:48:57.879 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:57.879 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:57.895 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:48:57.943 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:57.944+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:58.018 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:58.015+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:58.067 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:58.067 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:58.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:48:58.119 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:48:58.119 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:58.118+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:58.186 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:48:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:48:58.198 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:48:58.198 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:48:58.206 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-16T06:48:57.813687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":72778291,"num_objects":97,"num_object_clones":45,"num_object_copies":659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":6,"num_read":400,"num_read_kb":66753,"num_write":595,"num_write_kb":76246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":12222464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":912,"ondisk_log_size":912,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":575528,"kb_used_data":137392,"kb_used_omap":23,"kb_used_meta":437864,"kb_avail":1415002072,"statfs":{"total":1449551462400,"available":1448962121728,"internally_reserved":0,"allocated":140689408,"data_stored":175016776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23910,"internal_metadata":448373402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5718016,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":-1,"num_read":11,"num_read_kb":5285,"num_write":22,"num_write_kb":6496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.929718"},"pg_stats":[{"pgid":"3.f","version":"65'80","reported_seq":210,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.824282+0000","last_change":"2024-09-16T06:48:56.825226+0000","last_active":"2024-09-16T06:48:56.824282+0000","last_peered":"2024-09-16T06:48:56.824282+0000","last_clean":"2024-09-16T06:48:56.824282+0000","last_became_active":"2024-09-16T06:48:25.934897+0000","last_became_peered":"2024-09-16T06:48:25.934897+0000","last_unstale":"2024-09-16T06:48:56.824282+0000","last_undegraded":"2024-09-16T06:48:56.824282+0000","last_fullsized":"2024-09-16T06:48:56.824282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:30.322635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037462699999999999,"stat_sum":{"num_bytes":7241728,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":30,"num_read_kb":4477,"num_write":33,"num_write_kb":7184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1376256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,2,3,1],"acting":[11,12,4,14,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.e","version":"64'15","reported_seq":129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:54.921499+0000","last_change":"2024-09-16T06:48:51.836665+0000","last_active":"2024-09-16T06:48:54.921499+0000","last_peered":"2024-09-16T06:48:54.921499+0000","last_clean":"2024-09-16T06:48:54.921499+0000","last_became_active":"2024-09-16T06:48:25.920823+0000","last_became_peered":"2024-09-16T06:48:25.920823+0000","last_unstale":"2024-09-16T06:48:54.921499+0000","last_undegraded":"2024-09-16T06:48:54.921499+0000","last_fullsized":"2024-09-16T06:48:54.921499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.198332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019090468999999999,"stat_sum":{"num_bytes":1802240,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":5283,"num_write":6,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":294912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,2,0,5],"acting":[8,14,6,11,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.d","version":"64'55","reported_seq":179,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T06:48:52.830968+0000","last_change":"2024-09-16T06:48:51.858172+0000","last_active":"2024-09-16T06:48:52.830968+0000","last_peered":"2024-09-16T06:48:52.830968+0000","last_clean":"2024-09-16T06:48:52.830968+0000","last_became_active":"2024-09-16T06:48:42.825808+0000","last_became_peered":"2024-09-16T06:48:42.825808+0000","last_unstale":"2024-09-16T06:48:52.830968+0000","last_undegraded":"2024-09-16T06:48:52.830968+0000","last_fullsized":"2024-09-16T06:48:52.830968+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.854091+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.029640045,"stat_sum":{"num_bytes":4833280,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":2839,"num_write":21,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1015808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,8,5],"acting":[7,3,0,12,9,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.c","version":"69'78","reported_seq":197,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.841536+0000","last_change":"2024-09-16T06:48:56.828747+0000","last_active":"2024-09-16T06:48:56.841536+0000","last_peered":"2024-09-16T06:48:56.841536+0000","last_clean":"2024-09-16T06:48:56.841536+0000","last_became_active":"2024-09-16T06:48:25.936559+0000","last_became_peered":"2024-09-16T06:48:25.936559+0000","last_unstale":"2024-09-16T06:48:56.841536+0000","last_undegraded":"2024-09-16T06:48:56.841536+0000","last_fullsized":"2024-09-16T06:48:56.841536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:25.745881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083493779999999993,"stat_sum":{"num_bytes":9322496,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":17,"num_read_kb":3945,"num_write":31,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2441216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5,12,4,1,9],"acting":[11,2,5,12,4,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.b","version":"64'15","reported_seq":113,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.327667+0000","last_change":"2024-09-16T06:48:51.848473+0000","last_active":"2024-09-16T06:48:55.327667+0000","last_peered":"2024-09-16T06:48:55.327667+0000","last_clean":"2024-09-16T06:48:55.327667+0000","last_became_active":"2024-09-16T06:48:25.932423+0000","last_became_peered":"2024-09-16T06:48:25.932423+0000","last_unstale":"2024-09-16T06:48:55.327667+0000","last_undegraded":"2024-09-16T06:48:55.327667+0000","last_fullsized":"2024-09-16T06:48:55.327667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:30.178628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026397495,"stat_sum":{"num_bytes":344064,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":625,"num_write":6,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":311296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,10,3,8,7,14],"acting":[5,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.a","version":"64'109","reported_seq":193,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T06:48:52.830936+0000","last_change":"2024-09-16T06:48:51.887068+0000","last_active":"2024-09-16T06:48:52.830936+0000","last_peered":"2024-09-16T06:48:52.830936+0000","last_clean":"2024-09-16T06:48:52.830936+0000","last_became_active":"2024-09-16T06:48:20.885805+0000","last_became_peered":"2024-09-16T06:48:20.885805+0000","last_unstale":"2024-09-16T06:48:52.830936+0000","last_undegraded":"2024-09-16T06:48:52.830936+0000","last_fullsized":"2024-09-16T06:48:52.830936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:22.911175+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.056551470999999999,"stat_sum":{"num_bytes":8929280,"num_objects":11,"num_object_clones":5,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":24,"num_read_kb":3180,"num_write":37,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.9","version":"44'10","reported_seq":102,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:54.892364+0000","last_change":"2024-09-16T06:48:51.847517+0000","last_active":"2024-09-16T06:48:54.892364+0000","last_peered":"2024-09-16T06:48:54.892364+0000","last_clean":"2024-09-16T06:48:54.892364+0000","last_became_active":"2024-09-16T06:48:21.124872+0000","last_became_peered":"2024-09-16T06:48:21.124872+0000","last_unstale":"2024-09-16T06:48:54.892364+0000","last_undegraded":"2024-09-16T06:48:54.892364+0000","last_fullsized":"2024-09-16T06:48:54.892364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:43.972532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063603959999999999,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.8","version":"64'20","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:54.899461+0000","last_change":"2024-09-16T06:48:51.855620+0000","last_active":"2024-09-16T06:48:54.899461+0000","last_peered":"2024-09-16T06:48:54.899461+0000","last_clean":"2024-09-16T06:48:54.899461+0000","last_became_active":"2024-09-16T06:48:21.134991+0000","last_became_peered":"2024-09-16T06:48:21.134991+0000","last_unstale":"2024-09-16T06:48:54.899461+0000","last_undegraded":"2024-09-16T06:48:54.899461+0000","last_fullsized":"2024-09-16T06:48:54.899461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.487012+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.027669861,"stat_sum":{"num_bytes":2080768,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":5,"num_read_kb":3,"num_write":8,"num_write_kb":2032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,5,15,2],"acting":[6,1,0,4,5,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"1.0","version":"28'157","reported_seq":249,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:54.899729+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:48:54.899729+0000","last_peered":"2024-09-16T06:48:54.899729+0000","last_clean":"2024-09-16T06:48:54.899729+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:48:54.899729+0000","last_undegraded":"2024-09-16T06:48:54.899729+0000","last_fullsized":"2024-09-16T06:48:54.899729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.138478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.968329+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:48:55.968329+0000","last_peered":"2024-09-16T06:48:55.968329+0000","last_clean":"2024-09-16T06:48:55.968329+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:48:55.968329+0000","last_undegraded":"2024-09-16T06:48:55.968329+0000","last_fullsized":"2024-09-16T06:48:55.968329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"69'88","reported_seq":185,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.848244+0000","last_change":"2024-09-16T06:48:56.849882+0000","last_active":"2024-09-16T06:48:56.848244+0000","last_peered":"2024-09-16T06:48:56.848244+0000","last_clean":"2024-09-16T06:48:56.848244+0000","last_became_active":"2024-09-16T06:48:20.888283+0000","last_became_peered":"2024-09-16T06:48:20.888283+0000","last_unstale":"2024-09-16T06:48:56.848244+0000","last_undegraded":"2024-09-16T06:48:56.848244+0000","last_fullsized":"2024-09-16T06:48:56.848244+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.236427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024891519000000001,"stat_sum":{"num_bytes":5373952,"num_objects":13,"num_object_clones":8,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":22,"num_read_kb":6058,"num_write":65,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,10,11],"acting":[14,12,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T06:48:52.830625+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:48:52.830625+0000","last_peered":"2024-09-16T06:48:52.830625+0000","last_clean":"2024-09-16T06:48:52.830625+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:48:52.830625+0000","last_undegraded":"2024-09-16T06:48:52.830625+0000","last_fullsized":"2024-09-16T06:48:52.830625+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:16.422076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"69'32","reported_seq":127,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.842975+0000","last_change":"2024-09-16T06:48:56.844353+0000","last_active":"2024-09-16T06:48:56.842975+0000","last_peered":"2024-09-16T06:48:56.842975+0000","last_clean":"2024-09-16T06:48:56.842975+0000","last_became_active":"2024-09-16T06:48:25.928312+0000","last_became_peered":"2024-09-16T06:48:25.928312+0000","last_unstale":"2024-09-16T06:48:56.842975+0000","last_undegraded":"2024-09-16T06:48:56.842975+0000","last_fullsized":"2024-09-16T06:48:56.842975+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:32.643635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021197720999999999,"stat_sum":{"num_bytes":3457024,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":16,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":671744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,14,10,12,0,2],"acting":[1,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.968435+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:48:55.968435+0000","last_peered":"2024-09-16T06:48:55.968435+0000","last_clean":"2024-09-16T06:48:55.968435+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:48:55.968435+0000","last_undegraded":"2024-09-16T06:48:55.968435+0000","last_fullsized":"2024-09-16T06:48:55.968435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"64'44","reported_seq":136,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:54.892255+0000","last_change":"2024-09-16T06:48:51.865002+0000","last_active":"2024-09-16T06:48:54.892255+0000","last_peered":"2024-09-16T06:48:54.892255+0000","last_clean":"2024-09-16T06:48:54.892255+0000","last_became_active":"2024-09-16T06:48:21.124782+0000","last_became_peered":"2024-09-16T06:48:21.124782+0000","last_unstale":"2024-09-16T06:48:54.892255+0000","last_undegraded":"2024-09-16T06:48:54.892255+0000","last_fullsized":"2024-09-16T06:48:54.892255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.918920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.025632881,"stat_sum":{"num_bytes":2916352,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":12,"num_read_kb":773,"num_write":15,"num_write_kb":3168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T06:48:52.831179+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:48:52.831179+0000","last_peered":"2024-09-16T06:48:52.831179+0000","last_clean":"2024-09-16T06:48:52.831179+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:48:52.831179+0000","last_undegraded":"2024-09-16T06:48:52.831179+0000","last_fullsized":"2024-09-16T06:48:52.831179+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"64'78","reported_seq":183,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.827987+0000","last_change":"2024-09-16T06:48:56.831007+0000","last_active":"2024-09-16T06:48:56.827987+0000","last_peered":"2024-09-16T06:48:56.827987+0000","last_clean":"2024-09-16T06:48:56.827987+0000","last_became_active":"2024-09-16T06:48:21.121223+0000","last_became_peered":"2024-09-16T06:48:21.121223+0000","last_unstale":"2024-09-16T06:48:56.827987+0000","last_undegraded":"2024-09-16T06:48:56.827987+0000","last_fullsized":"2024-09-16T06:48:56.827987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.389211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035823439999999999,"stat_sum":{"num_bytes":7028736,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":33,"num_read_kb":3757,"num_write":50,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.5","version":"35'2","reported_seq":122,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.839479+0000","last_change":"2024-09-16T06:48:56.840615+0000","last_active":"2024-09-16T06:48:56.839479+0000","last_peered":"2024-09-16T06:48:56.839479+0000","last_clean":"2024-09-16T06:48:56.839479+0000","last_became_active":"2024-09-16T06:48:25.935381+0000","last_became_peered":"2024-09-16T06:48:25.935381+0000","last_unstale":"2024-09-16T06:48:56.839479+0000","last_undegraded":"2024-09-16T06:48:56.839479+0000","last_fullsized":"2024-09-16T06:48:56.839479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:48.456825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002504339,"stat_sum":{"num_bytes":475136,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":1394,"num_write":2,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":475136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0,5,2,7,1,8],"acting":[15,0,5,2,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.920647+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:48:55.920647+0000","last_peered":"2024-09-16T06:48:55.920647+0000","last_clean":"2024-09-16T06:48:55.920647+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:48:55.920647+0000","last_undegraded":"2024-09-16T06:48:55.920647+0000","last_fullsized":"2024-09-16T06:48:55.920647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"69'73","reported_seq":203,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T06:48:56.844339+0000","last_change":"2024-09-16T06:48:56.845898+0000","last_active":"2024-09-16T06:48:56.844339+0000","last_peered":"2024-09-16T06:48:56.844339+0000","last_clean":"2024-09-16T06:48:56.844339+0000","last_became_active":"2024-09-16T06:48:25.927267+0000","last_became_peered":"2024-09-16T06:48:25.927267+0000","last_unstale":"2024-09-16T06:48:56.844339+0000","last_undegraded":"2024-09-16T06:48:56.844339+0000","last_fullsized":"2024-09-16T06:48:56.844339+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:48.191646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022003953,"stat_sum":{"num_bytes":8519680,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":31,"num_read_kb":19115,"num_write":45,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1982464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,2,6,10,0],"acting":[1,12,9,2,6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:54.897768+0000","last_change":"2024-09-16T06:48:12.417012+0000","last_active":"2024-09-16T06:48:54.897768+0000","last_peered":"2024-09-16T06:48:54.897768+0000","last_clean":"2024-09-16T06:48:54.897768+0000","last_became_active":"2024-09-16T06:48:09.894853+0000","last_became_peered":"2024-09-16T06:48:09.894853+0000","last_unstale":"2024-09-16T06:48:54.897768+0000","last_undegraded":"2024-09-16T06:48:54.897768+0000","last_fullsized":"2024-09-16T06:48:54.897768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:26.719521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"65'49","reported_seq":148,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.111466+0000","last_change":"2024-09-16T06:48:51.847181+0000","last_active":"2024-09-16T06:48:55.111466+0000","last_peered":"2024-09-16T06:48:55.111466+0000","last_clean":"2024-09-16T06:48:55.111466+0000","last_became_active":"2024-09-16T06:48:25.935826+0000","last_became_peered":"2024-09-16T06:48:25.935826+0000","last_unstale":"2024-09-16T06:48:55.111466+0000","last_undegraded":"2024-09-16T06:48:55.111466+0000","last_fullsized":"2024-09-16T06:48:55.111466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:48.219953+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.031834551000000003,"stat_sum":{"num_bytes":8142848,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":27,"num_read_kb":8362,"num_write":47,"num_write_kb":8176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,6,1,14,10,11],"acting":[4,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.926559+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:48:55.926559+0000","last_peered":"2024-09-16T06:48:55.926559+0000","last_clean":"2024-09-16T06:48:55.926559+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:48:55.926559+0000","last_undegraded":"2024-09-16T06:48:55.926559+0000","last_fullsized":"2024-09-16T06:48:55.926559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"35'4","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T06:48:52.831013+0000","last_change":"2024-09-16T06:48:51.828719+0000","last_active":"2024-09-16T06:48:52.831013+0000","last_peered":"2024-09-16T06:48:52.831013+0000","last_clean":"2024-09-16T06:48:52.831013+0000","last_became_active":"2024-09-16T06:48:25.934152+0000","last_became_peered":"2024-09-16T06:48:25.934152+0000","last_unstale":"2024-09-16T06:48:52.831013+0000","last_undegraded":"2024-09-16T06:48:52.831013+0000","last_fullsized":"2024-09-16T06:48:52.831013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:28.717598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160035,"stat_sum":{"num_bytes":1163264,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":6820,"num_write":4,"num_write_kb":1136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-16T06:48:55.984040+0000","last_change":"2024-09-16T06:48:13.218911+0000","last_active":"2024-09-16T06:48:55.984040+0000","last_peered":"2024-09-16T06:48:55.984040+0000","last_clean":"2024-09-16T06:48:55.984040+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:48:55.984040+0000","last_undegraded":"2024-09-16T06:48:55.984040+0000","last_fullsized":"2024-09-16T06:48:55.984040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:09.738021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":72187904,"num_objects":93,"num_object_clones":45,"num_object_copies":651,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":6,"num_read":264,"num_read_kb":66635,"num_write":390,"num_write_kb":72784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":12222464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":128897024,"data_stored":170262528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247820,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38732,"kb_used_data":7608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333108,"statfs":{"total":96636764160,"available":96597102592,"internally_reserved":0,"allocated":7790592,"data_stored":9386218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,5,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.663}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7450000000000001}]}]},{"osd":14,"up_from":23,"seq":98784247822,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39492,"kb_used_data":8376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332348,"statfs":{"total":96636764160,"available":96596324352,"internally_reserved":0,"allocated":8577024,"data_stored":11018595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]}]},{"osd":11,"up_from":23,"seq":98784247821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43572,"kb_used_data":12448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328268,"statfs":{"total":96636764160,"available":96592146432,"internally_reserved":0,"allocated":12746752,"data_stored":16881898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":3,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36376,"kb_used_data":9356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335464,"statfs":{"total":96636764160,"available":96599515136,"internally_reserved":0,"allocated":9580544,"data_stored":11487467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.589}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7290000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.21}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8939999999999999}]}]},{"osd":13,"up_from":0,"seq":90194313229,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":23,"seq":98784247822,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37952,"kb_used_data":6832,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333888,"statfs":{"total":96636764160,"available":96597901312,"internally_reserved":0,"allocated":6995968,"data_stored":7392010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.081}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9570000000000001}]}]},{"osd":4,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":10616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":10870784,"data_stored":14569697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":5448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":5578752,"data_stored":6660321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,8,10,11,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1579999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]}]},{"osd":6,"up_from":22,"seq":94489280525,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36248,"kb_used_data":9220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335592,"statfs":{"total":96636764160,"available":96599646208,"internally_reserved":0,"allocated":9441280,"data_stored":10994433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]},{"osd":7,"up_from":22,"seq":94489280527,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33428,"kb_used_data":6404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":6557696,"data_stored":7250145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,5,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":8,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33856,"kb_used_data":6832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337984,"statfs":{"total":96636764160,"available":96602095616,"internally_reserved":0,"allocated":6995968,"data_stored":8241377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,5,6,7,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7749999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.347}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7089999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3690000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2850000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.169}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.399}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6320000000000001}]}]},{"osd":12,"up_from":22,"seq":94489280526,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38196,"kb_used_data":11168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333644,"statfs":{"total":96636764160,"available":96597651456,"internally_reserved":0,"allocated":11436032,"data_stored":14506218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.804}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.762}]}]},{"osd":1,"up_from":23,"seq":98784247823,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43852,"kb_used_data":12728,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327988,"statfs":{"total":96636764160,"available":96591859712,"internally_reserved":0,"allocated":13033472,"data_stored":16533738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":493.76100000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":2,"up_from":23,"seq":98784247821,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42940,"kb_used_data":11824,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328900,"statfs":{"total":96636764160,"available":96592793600,"internally_reserved":0,"allocated":12107776,"data_stored":14872931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]}]},{"osd":9,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38380,"kb_used_data":7264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333460,"statfs":{"total":96636764160,"available":96597463040,"internally_reserved":0,"allocated":7438336,"data_stored":10674550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.952}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.04}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3889999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9950000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1059999999999999}]}]},{"osd":10,"up_from":23,"seq":98784247821,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42392,"kb_used_data":11268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94329448,"statfs":{"total":96636764160,"available":96593354752,"internally_reserved":0,"allocated":11538432,"data_stored":14547178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,5,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5513216,"data_stored":6336512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12083200,"data_stored":16457728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11141120,"data_stored":14786560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8658944,"data_stored":10981376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9936896,"data_stored":14053376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4644864,"data_stored":6144000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8388608,"data_stored":10477568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5623808,"data_stored":6733824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6062080,"data_stored":7725056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":6467584,"data_stored":8908800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10588160,"data_stored":14172160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":11796480,"data_stored":16957440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":13979648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3055616,"data_stored":3055616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7610368,"data_stored":10481664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":6840320,"data_stored":9011200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:48:58.210 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-16T06:48:58.214 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:48:58.260 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:48:58.262+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:48:58.274 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:48:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:48:58.296 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:48:58.297 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:48:58.325 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:48:58.325 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:58.319+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:58.325 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:48:58.346 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:58.346 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:58.346 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:58.346 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:58.348 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:48:58.348 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:48:58.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:48:58.368 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:48:58.368 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:48:58.390 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:48:58.390 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:48:58.414 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:48:58.414 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:48:58.422 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:48:58.423+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:48:58.440 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:48:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:48:58.465 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:48:58.465 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:48:58.492 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:48:58.492 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:48:58.522 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:48:58.522 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:48:58.523 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:48:58.522+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:48:58.526 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:58.526 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:58.540 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:48:58.551 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:48:58.551 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:48:58.580 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:48:58.580 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:48:58.608 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:48:58.608 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:48:58.623 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:58.624+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:48:58.640+0000 7f405bfff640 0 -- 172.21.15.100:0/812045520 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7f405c21ba30 msgr2=0x7f405c21de20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:48:58.648 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:48:58.679 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:48:58.679 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:10 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.813276201000001, 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025558260999999999, 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896131+0000", 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896384+0000", 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025249400000000003 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896423+0000", 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9551999999999997e-05 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896446+0000", 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2657999999999999e-05 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896601+0000", 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015511400000000001 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896938+0000", 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000336779 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899233+0000", 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022945840000000001 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903156+0000", 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039230970000000004 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921526+0000", 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018370496 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921693+0000", 2024-09-16T06:48:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000166916 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:11 3.4s0 3:2be2ffb2:::smithi11731613-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.812923597000001, 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025954171000000002, 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896485+0000", 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896497+0000", 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1669e-05 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896518+0000", 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1180999999999999e-05 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896534+0000", 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5732000000000001e-05 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900249+0000", 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037148250000000002 2024-09-16T06:48:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900475+0000", 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022637300000000001 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901657+0000", 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001182249 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.909750+0000", 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0080923149999999992 2024-09-16T06:48:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922220+0000", 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012470623 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922442+0000", 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022146399999999999 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:12 3.4s0 3:2be2ffb2:::smithi11731613-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.812822236999999, 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032662215000000001, 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 12 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896587+0000", 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:58.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896595+0000", 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9389999999999997e-06 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896616+0000", 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0571e-05 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896631+0000", 2024-09-16T06:48:58.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5315000000000002e-05 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057790200000000002 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902594+0000", 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018354499999999999 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922598+0000", 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020004152000000001 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923951+0000", 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001352771 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929251+0000", 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005300389 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:34 3.4s0 3:2123e90c:::smithi11731613-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.808319957000002, 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031689311999999997, 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 34 2024-09-16T06:48:58.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901089+0000", 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901956+0000", 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086772399999999995 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901994+0000", 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7880000000000003e-05 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902023+0000", 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8479999999999998e-05 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902695+0000", 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067224400000000001 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902713+0000", 2024-09-16T06:48:58.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7719000000000001e-05 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.910786+0000", 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00807276 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911114+0000", 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032852700000000001 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914294+0000", 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003180113 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918604+0000", 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043102669999999996 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932586+0000", 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013981214 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932780+0000", 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019474 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:35 3.4s0 3:2123e90c:::smithi11731613-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.807365238000003, 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031230615999999999, 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 35 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902044+0000", 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902055+0000", 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0961999999999999e-05 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902075+0000", 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9748000000000001e-05 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902088+0000", 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3791999999999999e-05 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001041583 2024-09-16T06:48:58.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903146+0000", 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6511000000000002e-05 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.915912+0000", 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012765472999999999 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916161+0000", 2024-09-16T06:48:58.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024903700000000002 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917305+0000", 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001143813 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927287+0000", 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099818019999999997 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933082+0000", 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057955740000000004 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933276+0000", 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019429399999999999 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:36 3.4s0 3:2123e90c:::smithi11731613-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:58.740 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.806943177999997, 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038105068999999998, 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 36 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.742 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902465+0000", 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902479+0000", 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3405e-05 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902529+0000", 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9638999999999997e-05 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902555+0000", 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903331+0000", 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077605999999999997 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903346+0000", 2024-09-16T06:48:58.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5362999999999998e-05 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918677+0000", 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015331205000000001 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919160+0000", 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00048271000000000002 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936255+0000", 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017094597999999999 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936424+0000", 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168789 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940573+0000", 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041493600000000004 2024-09-16T06:48:58.746 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:228 3.4s0 3:2be2ffb2:::smithi11731613-4:head [rollback 5] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.751058661999998, 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015696132000000002, 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 228 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958355+0000", 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3280000000000002e-06 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958364+0000", 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6410000000000008e-06 2024-09-16T06:48:58.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958370+0000", 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0009999999999998e-06 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958705+0000", 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033533 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958720+0000", 2024-09-16T06:48:58.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4976e-05 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961715+0000", 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029947229999999999 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961796+0000", 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1070999999999997e-05 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962213+0000", 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00041693299999999997 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964999+0000", 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027864159999999999 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974015+0000", 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090157589999999999 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974049+0000", 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3501999999999999e-05 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:229 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:58.753 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.750916955000001, 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015649126999999999, 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 229 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.754 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958497+0000", 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4709999999999999e-06 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958504+0000", 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1500000000000002e-06 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958510+0000", 2024-09-16T06:48:58.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9399999999999999e-06 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958785+0000", 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027471700000000001 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958790+0000", 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9470000000000002e-06 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962470+0000", 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036807070000000001 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962542+0000", 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1168999999999996e-05 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962912+0000", 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037075200000000002 2024-09-16T06:48:58.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965035+0000", 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021227419999999999 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974112+0000", 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090767599999999997 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974143+0000", 2024-09-16T06:48:58.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1315000000000001e-05 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:237 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.750892050000001, 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013107461000000001, 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.760 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 237 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958519+0000", 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958521+0000", 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6469999999999998e-06 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.762 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958529+0000", 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8830000000000006e-06 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958535+0000", 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4410000000000001e-06 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958587+0000", 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2215000000000001e-05 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958594+0000", 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8589999999999999e-06 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960575+0000", 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019818380000000001 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.960703+0000", 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012750899999999999 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961363+0000", 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00066002300000000003 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964110+0000", 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027467659999999999 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.971588+0000", 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074782549999999996 2024-09-16T06:48:58.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.971626+0000", 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8476999999999998e-05 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:238 3.1s0 3:825f7e25:::smithi11731613-3:head [rollback 2] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.750866735999999, 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014029764, 2024-09-16T06:48:58.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 238 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958544+0000", 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958547+0000", 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2629999999999999e-06 2024-09-16T06:48:58.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958554+0000", 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0480000000000001e-06 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958561+0000", 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2919999999999998e-06 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958670+0000", 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010880800000000001 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958682+0000", 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1966e-05 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961922+0000", 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032400459999999999 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961990+0000", 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7710000000000001e-05 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962713+0000", 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000722738 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965012+0000", 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022997120000000002 2024-09-16T06:48:58.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972544+0000", 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007531055 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972574+0000", 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0550999999999999e-05 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:239 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.750844729000001, 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014095699999999999, 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 239 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.773 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958570+0000", 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7950000000000001e-06 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958576+0000", 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958583+0000", 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2629999999999997e-06 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958766+0000", 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018306299999999999 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958783+0000", 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6801000000000001e-05 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963004+0000", 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004221168 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963078+0000", 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3843000000000004e-05 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963329+0000", 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025082499999999998 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965054+0000", 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017253220000000001 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972634+0000", 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075798469999999998 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972662+0000", 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8232999999999999e-05 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:243 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.727433115, 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01277557, 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 243 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.779 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981978+0000", 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981977+0000", 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:58.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981982+0000", 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0669999999999999e-06 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.981993+0000", 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0535e-05 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.982010+0000", 2024-09-16T06:48:58.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7210999999999998e-05 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.984848+0000", 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028378570000000001 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.984935+0000", 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7505999999999999e-05 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.989994+0000", 2024-09-16T06:48:58.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050582639999999998 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990209+0000", 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021580699999999999 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994753+0000", 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045440899999999998 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:289 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.606003391000002, 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013340714, 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 289 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103408+0000", 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103406+0000", 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:58.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103414+0000", 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7219999999999999e-06 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103437+0000", 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2785e-05 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.103449+0000", 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2629000000000001e-05 2024-09-16T06:48:58.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.108874+0000", 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054250820000000003 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.109046+0000", 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000171173 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.109782+0000", 2024-09-16T06:48:58.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073599699999999995 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.111135+0000", 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013535349999999999 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.116590+0000", 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054547659999999998 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.116748+0000", 2024-09-16T06:48:58.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015846100000000001 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:493 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.820729+0000", 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.8886824699999991, 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021206388999999999, 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 493 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820729+0000", 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820729+0000", 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820725+0000", 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820743+0000", 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7316000000000002e-05 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820778+0000", 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5271999999999997e-05 2024-09-16T06:48:58.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820795+0000", 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7469000000000002e-05 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820976+0000", 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018091799999999999 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820989+0000", 2024-09-16T06:48:58.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2758999999999999e-05 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.825101+0000", 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041114259999999996 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.825255+0000", 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015408899999999999 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.841935+0000", 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016680326999999998 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:494 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.820816+0000", 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.8885948540000008, 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025957694999999999, 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:58.794 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 494 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820816+0000", 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820816+0000", 2024-09-16T06:48:58.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820815+0000", 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820821+0000", 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8660000000000002e-06 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820832+0000", 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0429e-05 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820840+0000", 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1799999999999996e-06 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821142+0000", 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030235899999999998 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821167+0000", 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4868e-05 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.826749+0000", 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055818980000000001 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.826925+0000", 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017605100000000001 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.846774+0000", 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019848973999999998 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.799 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:495 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.820881+0000", 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.8885298440000007, 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029233947999999999, 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 495 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820881+0000", 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820881+0000", 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820880+0000", 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820886+0000", 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4199999999999998e-06 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820911+0000", 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4825e-05 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820919+0000", 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2900000000000002e-06 2024-09-16T06:48:58.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821210+0000", 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000290916 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821219+0000", 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7260000000000008e-06 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.831203+0000", 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099845200000000002 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.831476+0000", 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027300700000000002 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.850115+0000", 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018639038 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.804 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpReadReply(3.4s0 61/37 ECSubReadReply(tid=93, attrs_read=0))", 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.829474+0000", 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.8799367839999999, 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012614276000000001, 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.829474+0000", 2024-09-16T06:48:58.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.829474+0000", 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.829472+0000", 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.835990+0000", 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0065175830000000004 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.838652+0000", 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026622640000000001 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.838661+0000", 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0890000000000006e-06 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.838711+0000", 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9688000000000003e-05 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.842089+0000", 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033775649999999999 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpReadReply(3.4s0 62/37 ECSubReadReply(tid=95, attrs_read=0))", 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.837361+0000", 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.8720496610000001, 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012866437, 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.837361+0000", 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.837361+0000", 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.837361+0000", 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.838620+0000", 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012592910000000001 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.838763+0000", 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000143363 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.838778+0000", 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4873e-05 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.846979+0000", 2024-09-16T06:48:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0082011570000000006 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.850228+0000", 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032484419999999998 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 64/37 ECSubWriteReply(tid=102, last_complete=64'64, committed=1, applied=1))", 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:51.863092+0000", 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.8463192199999998, 2024-09-16T06:48:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.95453391200000004, 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863092+0000", 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863092+0000", 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863091+0000", 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863095+0000", 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6169999999999998e-06 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.815760+0000", 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.95266482200000002 2024-09-16T06:48:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.815779+0000", 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9579e-05 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.815914+0000", 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134721 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.817626+0000", 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017117 2024-09-16T06:48:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:501 3.4s0 3:2123e90c:::smithi11731613-12:head [copy-from ver 2] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:52.822239+0000", 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.8871718409999998, 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014484334999999999, 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 501 2024-09-16T06:48:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822239+0000", 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822239+0000", 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822237+0000", 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822250+0000", 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3813e-05 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822284+0000", 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3575000000000003e-05 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822308+0000", 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3949000000000001e-05 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822403+0000", 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4914999999999995e-05 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822582+0000", 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000179502 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.830939+0000", 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0083570699999999994 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.832645+0000", 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017055499999999999 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.836594+0000", 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039489670000000003 2024-09-16T06:48:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.836724+0000", 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000129677 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:58.821 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:58.823 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:48:58.823 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:58.819+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:58.826 INFO:teuthology.orchestra.run.smithi100.stderr:set pool 3 pgp_num to 16 2024-09-16T06:48:58.826 INFO:tasks.rados.rados.0.smithi117.stdout:212: done (0 left) 2024-09-16T06:48:58.826 INFO:tasks.rados.rados.0.smithi117.stdout:213: rollback oid 35 current snap is 24 2024-09-16T06:48:58.826 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 35 to 23 2024-09-16T06:48:58.827 INFO:tasks.rados.rados.0.smithi117.stdout:214: snap_remove snap 8 2024-09-16T06:48:58.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:48:58.924 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:48:58.924+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:48:58.957 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:58.957 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:58.957 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:58.957 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:58.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:48:59.025 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:48:59.023+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:48:59.092 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:59.092 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:59.092 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:59.092 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:59.092 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:59.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:48:59.126 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:59.123+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:59.227 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:48:59.228+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:48:59.233 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.es6 35/32 ECSubWrite(tid=1, reqid=client.4511.0:53, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.290054458999997, 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.67331382100000003, 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942604+0000", 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942602+0000", 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.942641+0000", 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9292999999999998e-05 2024-09-16T06:48:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614766+0000", 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.67212478200000003 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614782+0000", 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5654999999999998e-05 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614883+0000", 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010158 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.614901+0000", 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7337999999999999e-05 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.615860+0000", 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00095891399999999995 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.615918+0000", 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7978e-05 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/32 ECSubWrite(tid=160, reqid=client.4511.0:119, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.203683034000001, 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050875180000000001, 2024-09-16T06:48:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028975+0000", 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028974+0000", 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029016+0000", 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2521000000000002e-05 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029189+0000", 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017280600000000001 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029213+0000", 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4394999999999999e-05 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029795+0000", 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00058197499999999998 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.029816+0000", 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0650999999999999e-05 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.034007+0000", 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041915119999999997 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.034063+0000", 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5145999999999997e-05 2024-09-16T06:48:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=18, reqid=client.4511.0:130, at_version=35'1, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.187035061000003, 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051140320000000001, 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045623+0000", 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.045622+0000", 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046511+0000", 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00088898600000000003 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046635+0000", 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124404 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046683+0000", 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.761e-05 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046815+0000", 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013226000000000001 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.046828+0000", 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2502e-05 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050686+0000", 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038587869999999998 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.050737+0000", 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0785000000000003e-05 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=20, reqid=client.4511.0:131, at_version=0'0, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.181000304000001, 2024-09-16T06:48:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061083889999999997, 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051658+0000", 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051650+0000", 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999924 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051678+0000", 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8297999999999999e-05 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051750+0000", 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1749999999999996e-05 2024-09-16T06:48:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051777+0000", 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7169999999999999e-05 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051852+0000", 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4768e-05 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051866+0000", 2024-09-16T06:48:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3893000000000001e-05 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057731+0000", 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00586551 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.057766+0000", 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4795999999999997e-05 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 35/32 ECSubWrite(tid=31, reqid=client.4511.0:161, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.717449928000001, 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054402210000000003, 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515208+0000", 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515206+0000", 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515247+0000", 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0228999999999997e-05 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515342+0000", 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5420000000000005e-05 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515356+0000", 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3952e-05 2024-09-16T06:48:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516560+0000", 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012035839999999999 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516583+0000", 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3064000000000001e-05 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520594+0000", 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040113630000000004 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520648+0000", 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4425000000000002e-05 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:221 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.268996969, 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0098559149999999998, 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 221 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963661+0000", 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963659+0000", 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963670+0000", 2024-09-16T06:48:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0423e-05 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963685+0000", 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5596000000000001e-05 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963698+0000", 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2413e-05 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963842+0000", 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144444 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963862+0000", 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9874999999999999e-05 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.967541+0000", 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036789790000000002 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.967631+0000", 2024-09-16T06:48:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9548000000000004e-05 2024-09-16T06:48:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.973517+0000", 2024-09-16T06:48:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00588654 2024-09-16T06:48:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:222 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.268940164, 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010055573, 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 222 2024-09-16T06:48:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963718+0000", 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963717+0000", 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963728+0000", 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0609e-05 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963739+0000", 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0829000000000001e-05 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963762+0000", 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2894000000000001e-05 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024621499999999999 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964013+0000", 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7840000000000003e-06 2024-09-16T06:48:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968066+0000", 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040535229999999998 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968114+0000", 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7788999999999999e-05 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.973774+0000", 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056596190000000003 2024-09-16T06:48:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:223 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.268853744000001, 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011053090999999999, 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 223 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963804+0000", 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963803+0000", 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963818+0000", 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4983999999999999e-05 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963830+0000", 2024-09-16T06:48:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1472e-05 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963839+0000", 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2420000000000006e-06 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964057+0000", 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217455 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964066+0000", 2024-09-16T06:48:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9570000000000008e-06 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968321+0000", 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042553460000000001 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.968427+0000", 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105887 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974857+0000", 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064307560000000001 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds6 40/37 ECSubWrite(tid=21, reqid=client.4511.0:219, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.242310001, 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057451949999999998, 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990348+0000", 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990346+0000", 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990377+0000", 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0463999999999998e-05 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990469+0000", 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1932e-05 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990492+0000", 2024-09-16T06:48:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3127999999999999e-05 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990628+0000", 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000136212 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.990653+0000", 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5289000000000001e-05 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.996040+0000", 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053864309999999997 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.996093+0000", 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3735e-05 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 40/37 ECSubWrite(tid=59, reqid=client.4511.0:233, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.238936027000001, 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064058910000000004, 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993722+0000", 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.993721+0000", 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994930+0000", 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001209803 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995211+0000", 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028033799999999998 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995239+0000", 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8453999999999999e-05 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995377+0000", 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000137797 2024-09-16T06:48:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995407+0000", 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0154999999999998e-05 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000083+0000", 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046756920000000004 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000128+0000", 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5241999999999998e-05 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:286 3.bs0 3:db3fccb6:::smithi11731613-6:head [stat,delete] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.225225269999999, 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019136058000000001, 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 286 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007433+0000", 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007430+0000", 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007447+0000", 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7146000000000002e-05 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007483+0000", 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-05 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007508+0000", 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5701e-05 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007675+0000", 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016656 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.007699+0000", 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3742000000000001e-05 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012471+0000", 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047720949999999996 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.012760+0000", 2024-09-16T06:48:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028969899999999998 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.014682+0000", 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019216789999999999 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.017646+0000", 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029635830000000001 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.026466+0000", 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0088207990000000007 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:30.026569+0000", 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010246099999999999 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:39.858539+0000", 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.374118702000001, 2024-09-16T06:48:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051563729999999997, 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858539+0000", 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858539+0000", 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858538+0000", 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858605+0000", 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7175999999999995e-05 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858706+0000", 2024-09-16T06:48:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010049 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858754+0000", 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7854e-05 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858912+0000", 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000158243 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.858940+0000", 2024-09-16T06:48:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7915e-05 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.863635+0000", 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046950519999999999 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:39.863696+0000", 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0782999999999997e-05 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:426 3.bs0 3:db3fccb6:::smithi11731613-6:head [copy-from ver 2] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:40.856438+0000", 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.376220354000001, 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.63165375099999999, 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 426 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.296 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856438+0000", 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856438+0000", 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856435+0000", 2024-09-16T06:48:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856451+0000", 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6209e-05 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856487+0000", 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5737e-05 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856507+0000", 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0489e-05 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856665+0000", 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015755399999999999 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856691+0000", 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5955000000000001e-05 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864103+0000", 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074121389999999999 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864254+0000", 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015063000000000001 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.484545+0000", 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.62029161200000005 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.487326+0000", 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002780827 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.487444+0000", 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118403 2024-09-16T06:48:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.488092+0000", 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00064706500000000005 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:427 3.bs0 3:db3fccb6:::smithi11731613-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:40.856528+0000", 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.376130052000001, 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.62975156700000001, 2024-09-16T06:48:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 427 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856528+0000", 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856528+0000", 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856527+0000", 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856534+0000", 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1749999999999999e-06 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856556+0000", 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2469000000000001e-05 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856574+0000", 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7805999999999999e-05 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856805+0000", 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000230669 2024-09-16T06:48:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.856818+0000", 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3106e-05 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864540+0000", 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077218599999999997 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:40.864630+0000", 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0366999999999997e-05 2024-09-16T06:48:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.485781+0000", 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.621151224 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.486017+0000", 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000235436 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:41.486280+0000", 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000262828 2024-09-16T06:48:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/37 ECSubWrite(tid=160, reqid=osd.11.0:160, at_version=61'66, trim_to=0'0, roll_forward_to=58'62))", 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.831205+0000", 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.401453292999999, 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057872180000000002, 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831205+0000", 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831205+0000", 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831203+0000", 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831243+0000", 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9418000000000001e-05 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831310+0000", 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7130000000000003e-05 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831335+0000", 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4964999999999999e-05 2024-09-16T06:48:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831482+0000", 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014747299999999999 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831509+0000", 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6812e-05 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.836937+0000", 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054279089999999999 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.836992+0000", 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4901000000000002e-05 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.es6 61/37 ECSubWrite(tid=15, reqid=osd.8.0:15, at_version=61'13, trim_to=0'0, roll_forward_to=57'11))", 2024-09-16T06:48:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.833419+0000", 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.399238808, 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070228180000000001, 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833419+0000", 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833419+0000", 2024-09-16T06:48:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833419+0000", 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833475+0000", 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6888000000000001e-05 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833577+0000", 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102038 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833599+0000", 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1339e-05 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833740+0000", 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000140903 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833771+0000", 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1152000000000003e-05 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.840348+0000", 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006576976 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.840442+0000", 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4299e-05 2024-09-16T06:48:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:503 3.bs0 3:d4957fab:::smithi11731613-13:head [copy-from ver 0] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:52.822259+0000", 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.4103994919999998, 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050264680000000001, 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 503 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822259+0000", 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822259+0000", 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822255+0000", 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822272+0000", 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6821999999999999e-05 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822306+0000", 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3932e-05 2024-09-16T06:48:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822326+0000", 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9936e-05 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822501+0000", 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017448499999999999 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:48:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822522+0000", 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1688999999999999e-05 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.825948+0000", 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034252000000000002 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.826087+0000", 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013968699999999999 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.827285+0000", 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011979029999999999 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:504 3.bs0 3:d4957fab:::smithi11731613-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:52.822348+0000", 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.410310452, 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055497740000000004, 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:48:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 504 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822348+0000", 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822348+0000", 2024-09-16T06:48:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822347+0000", 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822352+0000", 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6930000000000004e-06 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822385+0000", 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2941999999999997e-05 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822401+0000", 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5196e-05 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822674+0000", 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000273078 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822693+0000", 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9809e-05 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.826331+0000", 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003637323 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for blocked object", 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.826417+0000", 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6452999999999994e-05 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.827425+0000", 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001007785 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.827622+0000", 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019714799999999999 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.827897+0000", 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000275271 2024-09-16T06:48:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.es6 71/37 ECSubWrite(tid=22, reqid=osd.8.0:22, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.835656+0000", 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.39700223899999998, 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0065232509999999999, 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835656+0000", 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835656+0000", 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835655+0000", 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835708+0000", 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3130999999999997e-05 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835862+0000", 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015378700000000001 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835884+0000", 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2019999999999999e-05 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836035+0000", 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015049099999999999 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836061+0000", 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5885999999999999e-05 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.842116+0000", 2024-09-16T06:48:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060549710000000001 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.842179+0000", 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3540999999999994e-05 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs2 71/37 ECSubWrite(tid=191, reqid=osd.11.0:191, at_version=71'82, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.835971+0000", 2024-09-16T06:48:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.39668699699999999, 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062498340000000001, 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835971+0000", 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835971+0000", 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.835970+0000", 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836014+0000", 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3702000000000003e-05 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836083+0000", 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9500999999999997e-05 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836106+0000", 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3224999999999999e-05 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836251+0000", 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014408499999999999 2024-09-16T06:48:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836277+0000", 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6509e-05 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.842190+0000", 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0059132109999999998 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.842221+0000", 2024-09-16T06:48:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0747999999999998e-05 2024-09-16T06:48:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.337 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:59.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:48:59.339 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:48:59.339+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:48:59.440 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:48:59.444+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:48:59.462 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:59.462 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:59.462 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:48:59.462 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:59.476 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:48:59.541 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:59.542+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:59.594 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:59.594 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:48:59.595 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:48:59.595 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:48:59.595 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:59.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:48:59.642 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:48:59.639+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:48:59.742 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:48:59.743+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:48:59.771 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:48:59.771 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:48:59.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.871062479000003, 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064369220000000003, 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:59.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899647+0000", 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.773 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899689+0000", 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2429000000000003e-05 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899768+0000", 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9277999999999995e-05 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899797+0000", 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8473999999999999e-05 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903489+0000", 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036918680000000001 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903519+0000", 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0283000000000001e-05 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906024+0000", 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025045430000000001 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906085+0000", 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1637000000000006e-05 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.721895449999998, 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052817840000000003, 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048815+0000", 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048832+0000", 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7638e-05 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048870+0000", 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8114000000000001e-05 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.778 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.048877+0000", 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4849999999999997e-06 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.049092+0000", 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000214806 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.779 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.049100+0000", 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8170000000000005e-06 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054030+0000", 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049292750000000003 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.054097+0000", 2024-09-16T06:48:59.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7453000000000006e-05 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 35/32 ECSubWrite(tid=10, reqid=client.4511.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.719543283999997, 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005244037, 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051167+0000", 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051166+0000", 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051182+0000", 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5571000000000001e-05 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051230+0000", 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7703000000000001e-05 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051243+0000", 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3573e-05 2024-09-16T06:48:59.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051327+0000", 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4098999999999996e-05 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.051343+0000", 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5812e-05 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056375+0000", 2024-09-16T06:48:59.784 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050313950000000001 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.056411+0000", 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6890000000000001e-05 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=30, reqid=client.4511.0:160, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.256061926000001, 2024-09-16T06:48:59.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056229460000000002, 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514649+0000", 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.514647+0000", 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515002+0000", 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000354607 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515104+0000", 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102607 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515128+0000", 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3998e-05 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515678+0000", 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055015800000000003 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515703+0000", 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4706999999999999e-05 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520193+0000", 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044896160000000001 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520272+0000", 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9089000000000001e-05 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 35/32 ECSubWrite(tid=31, reqid=client.4511.0:161, at_version=35'9, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:48:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.255532922, 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056966849999999999, 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515178+0000", 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515177+0000", 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515216+0000", 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9765999999999998e-05 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515642+0000", 2024-09-16T06:48:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042582399999999999 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.515672+0000", 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9403e-05 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516695+0000", 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010230549999999999 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.516716+0000", 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1858e-05 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520824+0000", 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041076059999999998 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.520875+0000", 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0479000000000002e-05 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 40/37 ECSubWrite(tid=59, reqid=client.4511.0:233, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.776388043000001, 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061889309999999999, 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994322+0000", 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995376+0000", 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00105338 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995464+0000", 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8170000000000005e-05 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995483+0000", 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9548999999999999e-05 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995633+0000", 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014933200000000001 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995662+0000", 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8847999999999999e-05 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000463+0000", 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048011360000000001 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000512+0000", 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8860999999999999e-05 2024-09-16T06:48:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 47/37 ECSubWrite(tid=91, reqid=client.4511.0:355, at_version=47'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.924067383000001, 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061968889999999997, 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846717+0000", 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4350999999999993e-05 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846862+0000", 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014473599999999999 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846886+0000", 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4289000000000001e-05 2024-09-16T06:48:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.847145+0000", 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000259158 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.847176+0000", 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1154e-05 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.852788+0000", 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056113070000000003 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.852840+0000", 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2509999999999997e-05 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 59/32 ECSubWrite(tid=54, reqid=client.4511.0:469, at_version=59'36, trim_to=0'0, roll_forward_to=57'34))", 2024-09-16T06:48:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:46.841357+0000", 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.929353592, 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048609120000000002, 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.841357+0000", 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.841357+0000", 2024-09-16T06:48:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.841357+0000", 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.841874+0000", 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000517593 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.841935+0000", 2024-09-16T06:48:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1277000000000003e-05 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.841957+0000", 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1872e-05 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.842111+0000", 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015357899999999999 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.842143+0000", 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2027999999999999e-05 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.846140+0000", 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039973630000000003 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:46.846218+0000", 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7725999999999994e-05 2024-09-16T06:48:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 61/37 ECSubWrite(tid=90, reqid=osd.1.0:90, at_version=61'54, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.818460+0000", 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.952250746000001, 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055543759999999998, 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818460+0000", 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818460+0000", 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818459+0000", 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818509+0000", 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0074000000000002e-05 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818646+0000", 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0001371 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818676+0000", 2024-09-16T06:48:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9912000000000001e-05 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818864+0000", 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018794699999999999 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.818893+0000", 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8411999999999998e-05 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823958+0000", 2024-09-16T06:48:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050650900000000004 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.824014+0000", 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6623999999999999e-05 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 61/32 ECSubWrite(tid=56, reqid=osd.12.0:56, at_version=61'38, trim_to=0'0, roll_forward_to=59'36))", 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.820523+0000", 2024-09-16T06:48:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.95018726, 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060238599999999998, 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820523+0000", 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820523+0000", 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820523+0000", 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820608+0000", 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5389000000000005e-05 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820723+0000", 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114595 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820749+0000", 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5950999999999999e-05 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820888+0000", 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013880399999999999 2024-09-16T06:48:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.820916+0000", 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8402e-05 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.826478+0000", 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055619019999999996 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.826547+0000", 2024-09-16T06:48:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9437000000000006e-05 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.821439+0000", 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.949271905, 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051780979999999999, 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821439+0000", 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821439+0000", 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821437+0000", 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821490+0000", 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2948000000000002e-05 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821594+0000", 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103955 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821617+0000", 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2665000000000001e-05 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821801+0000", 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018459100000000001 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.821830+0000", 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8402e-05 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.826569+0000", 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047396690000000002 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.826617+0000", 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7657999999999997e-05 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 61/32 ECSubWrite(tid=57, reqid=osd.12.0:57, at_version=61'40, trim_to=0'0, roll_forward_to=59'36))", 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.947850074, 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068954469999999999, 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822859+0000", 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822915+0000", 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6230999999999997e-05 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823009+0000", 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4176000000000001e-05 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823032+0000", 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3005000000000001e-05 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823531+0000", 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000498326 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823611+0000", 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0495999999999996e-05 2024-09-16T06:48:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829687+0000", 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060763509999999998 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829756+0000", 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8765999999999999e-05 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/37 ECSubWrite(tid=158, reqid=osd.11.0:158, at_version=61'64, trim_to=0'0, roll_forward_to=58'62))", 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.828974+0000", 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.941736718, 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047164570000000003, 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.828974+0000", 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.828974+0000", 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.828974+0000", 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829011+0000", 2024-09-16T06:48:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7054999999999997e-05 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829120+0000", 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109511 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829142+0000", 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1753e-05 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829286+0000", 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000143676 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829312+0000", 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6219000000000001e-05 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833513+0000", 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042009819999999998 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.833690+0000", 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017758499999999999 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/37 ECSubWrite(tid=160, reqid=osd.11.0:160, at_version=61'66, trim_to=0'0, roll_forward_to=58'62))", 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.831270+0000", 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.939441147, 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051745849999999998, 2024-09-16T06:48:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831270+0000", 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831270+0000", 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831268+0000", 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831322+0000", 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4410000000000003e-05 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831443+0000", 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000121171 2024-09-16T06:48:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831468+0000", 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4468999999999999e-05 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831647+0000", 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017898799999999999 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.831674+0000", 2024-09-16T06:48:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7092000000000001e-05 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.836388+0000", 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047141099999999997 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.836444+0000", 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6070999999999999e-05 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.93757493700000005, 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069218320000000002, 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:48:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833135+0000", 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833193+0000", 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8183999999999997e-05 2024-09-16T06:48:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833314+0000", 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012045900000000001 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833341+0000", 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6954999999999999e-05 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833966+0000", 2024-09-16T06:48:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00062526400000000005 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833994+0000", 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7942000000000001e-05 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.840004+0000", 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060096969999999996 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.840058+0000", 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3931000000000003e-05 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 71/37 ECSubWrite(tid=191, reqid=osd.11.0:191, at_version=71'82, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.836506+0000", 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.93420439399999999, 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058666109999999999, 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836506+0000", 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836506+0000", 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836505+0000", 2024-09-16T06:48:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836544+0000", 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8754999999999997e-05 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836613+0000", 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9450999999999996e-05 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836635+0000", 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2269e-05 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836749+0000", 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114145 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.836774+0000", 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4686e-05 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.842345+0000", 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055711939999999998 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.842373+0000", 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7661000000000002e-05 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 71/32 ECSubWrite(tid=62, reqid=osd.12.0:62, at_version=71'46, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:48:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.841338+0000", 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.92937314599999998, 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047217780000000003, 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841338+0000", 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841338+0000", 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841337+0000", 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841371+0000", 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3728e-05 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841467+0000", 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5717000000000005e-05 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841476+0000", 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.867e-06 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841625+0000", 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000149016 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.841650+0000", 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5452e-05 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.845986+0000", 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043361659999999998 2024-09-16T06:48:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.846059+0000", 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3040999999999994e-05 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 71/32 ECSubWrite(tid=64, reqid=osd.12.0:63, at_version=0'0, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.873456+0000", 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.89725502400000001, 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047471930000000002, 2024-09-16T06:48:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873456+0000", 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873456+0000", 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873454+0000", 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873483+0000", 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8733000000000001e-05 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873570+0000", 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7473999999999996e-05 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873592+0000", 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1467999999999998e-05 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873665+0000", 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3543999999999999e-05 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.873675+0000", 2024-09-16T06:48:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9259999999999995e-06 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.878149+0000", 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044743580000000003 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.878203+0000", 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3504000000000001e-05 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 71/54 ECSubWrite(tid=154, reqid=osd.3.0:149, at_version=0'0, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.89512007000000005, 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076751120000000004, 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875589+0000", 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875617+0000", 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8209999999999999e-05 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875704+0000", 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7047999999999997e-05 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875725+0000", 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0837000000000001e-05 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875784+0000", 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9077999999999999e-05 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875794+0000", 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5319999999999995e-06 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883218+0000", 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074248020000000003 2024-09-16T06:48:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883266+0000", 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7339000000000002e-05 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 71/37 ECSubWrite(tid=129, reqid=osd.1.0:129, at_version=71'79, trim_to=0'0, roll_forward_to=71'77))", 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.89460443999999995, 2024-09-16T06:48:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071128010000000002, 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:48:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876105+0000", 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876130+0000", 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.497e-05 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876185+0000", 2024-09-16T06:48:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4926999999999998e-05 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876194+0000", 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7669999999999992e-06 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876307+0000", 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113189 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876326+0000", 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8553999999999999e-05 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883200+0000", 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006874092 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883219+0000", 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9286000000000001e-05 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:48:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:48:59.861 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:48:59.862 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:214: done (1 left) 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 14 current snap is 24 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:213: finishing rollback tid 0 to smithi11731613-35 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:213: finishing rollback tid 1 to smithi11731613-35 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:213: finishing rollback tid 2 to smithi11731613-35 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 35 v 8 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-16T06:48:59.863 INFO:tasks.rados.rados.0.smithi117.stdout:215: seq_num 100 ranges {1081344=524288,1605632=753664,2359296=409600} 2024-09-16T06:48:59.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:48:59.864 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:48:59.859+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:48:59.869 INFO:tasks.rados.rados.0.smithi117.stdout:215: writing smithi11731613-14 from 1081344 to 1605632 tid 1 2024-09-16T06:48:59.886 INFO:tasks.rados.rados.0.smithi117.stdout:215: writing smithi11731613-14 from 1605632 to 2359296 tid 2 2024-09-16T06:48:59.893 INFO:tasks.rados.rados.0.smithi117.stdout:215: writing smithi11731613-14 from 2359296 to 2768896 tid 3 2024-09-16T06:48:59.894 INFO:tasks.rados.rados.0.smithi117.stdout:213: done (1 left) 2024-09-16T06:48:59.894 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 37 current snap is 24 2024-09-16T06:48:59.894 INFO:tasks.rados.rados.0.smithi117.stdout:215: finishing write tid 1 to smithi11731613-14 2024-09-16T06:48:59.894 INFO:tasks.rados.rados.0.smithi117.stdout:216: seq_num 101 ranges {3768320=540672,4308992=507904,4816896=622592,5439488=32768} 2024-09-16T06:48:59.942 INFO:tasks.rados.rados.0.smithi117.stdout:216: writing smithi11731613-37 from 3768320 to 4308992 tid 1 2024-09-16T06:48:59.947 INFO:tasks.rados.rados.0.smithi117.stdout:216: writing smithi11731613-37 from 4308992 to 4816896 tid 2 2024-09-16T06:48:59.953 INFO:tasks.rados.rados.0.smithi117.stdout:216: writing smithi11731613-37 from 4816896 to 5439488 tid 3 2024-09-16T06:48:59.953 INFO:tasks.rados.rados.0.smithi117.stdout:216: writing smithi11731613-37 from 5439488 to 5472256 tid 4 2024-09-16T06:48:59.953 INFO:tasks.rados.rados.0.smithi117.stdout:217: read oid 29 snap 18 2024-09-16T06:48:59.953 INFO:tasks.rados.rados.0.smithi117.stdout:217: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-16T06:48:59.953 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 40 current snap is 24 2024-09-16T06:48:59.953 INFO:tasks.rados.rados.0.smithi117.stdout:215: finishing write tid 2 to smithi11731613-14 2024-09-16T06:48:59.954 INFO:tasks.rados.rados.0.smithi117.stdout:215: finishing write tid 3 to smithi11731613-14 2024-09-16T06:48:59.954 INFO:tasks.rados.rados.0.smithi117.stdout:215: finishing write tid 4 to smithi11731613-14 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:215: finishing write tid 5 to smithi11731613-14 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:215: oid 14 updating version 0 to 63 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:215: oid 14 updating version 63 to 64 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:215: oid 14 updating version 64 to 66 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:215: oid 14 version 66 is already newer than 65 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 14 v 66 (ObjNum 100 snap 24 seq_num 100) dirty exists 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:215: left oid 14 (ObjNum 100 snap 24 seq_num 100) 2024-09-16T06:48:59.956 INFO:tasks.rados.rados.0.smithi117.stdout:218: seq_num 102 ranges {671744=557056,1228800=638976,1867776=524288} 2024-09-16T06:48:59.963 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:48:59.962+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:48:59.963 INFO:tasks.rados.rados.0.smithi117.stdout:218: writing smithi11731613-40 from 671744 to 1228800 tid 1 2024-09-16T06:48:59.968 INFO:tasks.rados.rados.0.smithi117.stdout:218: writing smithi11731613-40 from 1228800 to 1867776 tid 2 2024-09-16T06:48:59.972 INFO:tasks.rados.rados.0.smithi117.stdout:218: writing smithi11731613-40 from 1867776 to 2392064 tid 3 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:215: done (3 left) 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:219: rollback oid 20 current snap is 24 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 20 to 23 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:220: read oid 31 snap -1 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:220: expect (ObjNum 73 snap 7 seq_num 73) 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 49 current snap is 24 2024-09-16T06:48:59.973 INFO:tasks.rados.rados.0.smithi117.stdout:221: seq_num 103 ranges {2457600=573440,3031040=786432,3817472=376832} 2024-09-16T06:48:59.995 INFO:tasks.rados.rados.0.smithi117.stdout:221: writing smithi11731613-49 from 2457600 to 3031040 tid 1 2024-09-16T06:49:00.000 INFO:tasks.rados.rados.0.smithi117.stdout:221: writing smithi11731613-49 from 3031040 to 3817472 tid 2 2024-09-16T06:49:00.001 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:00.001 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:00.001 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:49:00.001 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:00.003 INFO:tasks.rados.rados.0.smithi117.stdout:221: writing smithi11731613-49 from 3817472 to 4194304 tid 3 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:222: copy_from oid 43 from oid 35 current snap is 24 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:216: finishing write tid 1 to smithi11731613-37 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:216: finishing write tid 2 to smithi11731613-37 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:216: finishing write tid 3 to smithi11731613-37 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 34 current snap is 24 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:216: finishing write tid 4 to smithi11731613-37 2024-09-16T06:49:00.004 INFO:tasks.rados.rados.0.smithi117.stdout:217: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-16T06:49:00.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:49:00.043 INFO:tasks.rados.rados.0.smithi117.stdout:218: finishing write tid 1 to smithi11731613-40 2024-09-16T06:49:00.043 INFO:tasks.rados.rados.0.smithi117.stdout:218: finishing write tid 2 to smithi11731613-40 2024-09-16T06:49:00.043 INFO:tasks.rados.rados.0.smithi117.stdout:219: finishing rollback tid 1 to smithi11731613-20 2024-09-16T06:49:00.043 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 20 v 0 (ObjNum 129 snap 0 seq_num 1686865600) dirty dne 2024-09-16T06:49:00.044 INFO:tasks.rados.rados.0.smithi117.stdout:218: finishing write tid 3 to smithi11731613-40 2024-09-16T06:49:00.044 INFO:tasks.rados.rados.0.smithi117.stdout:218: finishing write tid 4 to smithi11731613-40 2024-09-16T06:49:00.044 INFO:tasks.rados.rados.0.smithi117.stdout:218: finishing write tid 5 to smithi11731613-40 2024-09-16T06:49:00.045 INFO:tasks.rados.rados.0.smithi117.stdout:218: oid 40 updating version 0 to 27 2024-09-16T06:49:00.045 INFO:tasks.rados.rados.0.smithi117.stdout:218: oid 40 updating version 27 to 29 2024-09-16T06:49:00.045 INFO:tasks.rados.rados.0.smithi117.stdout:218: oid 40 version 29 is already newer than 26 2024-09-16T06:49:00.045 INFO:tasks.rados.rados.0.smithi117.stdout:218: oid 40 version 29 is already newer than 28 2024-09-16T06:49:00.045 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 40 v 29 (ObjNum 102 snap 24 seq_num 102) dirty exists 2024-09-16T06:49:00.046 INFO:tasks.rados.rados.0.smithi117.stdout:218: left oid 40 (ObjNum 102 snap 24 seq_num 102) 2024-09-16T06:49:00.046 INFO:tasks.rados.rados.0.smithi117.stdout:223: seq_num 104 ranges {557056=589824,1146880=540672,1687552=622592} 2024-09-16T06:49:00.049 INFO:tasks.rados.rados.0.smithi117.stdout:223: writing smithi11731613-34 from 557056 to 1146880 tid 1 2024-09-16T06:49:00.053 INFO:tasks.rados.rados.0.smithi117.stdout:223: writing smithi11731613-34 from 1146880 to 1687552 tid 2 2024-09-16T06:49:00.057 INFO:tasks.rados.rados.0.smithi117.stdout:223: writing smithi11731613-34 from 1687552 to 2310144 tid 3 2024-09-16T06:49:00.057 INFO:tasks.rados.rados.0.smithi117.stdout:217: done (7 left) 2024-09-16T06:49:00.057 INFO:tasks.rados.rados.0.smithi117.stdout:218: done (6 left) 2024-09-16T06:49:00.057 INFO:tasks.rados.rados.0.smithi117.stdout:219: done (5 left) 2024-09-16T06:49:00.057 INFO:tasks.rados.rados.0.smithi117.stdout:224: rollback oid 7 current snap is 24 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 7 to 21 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: finishing write tid 5 to smithi11731613-37 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: finishing write tid 6 to smithi11731613-37 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: oid 37 updating version 0 to 72 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: oid 37 version 72 is already newer than 69 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: oid 37 version 72 is already newer than 70 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: oid 37 version 72 is already newer than 71 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: oid 37 version 72 is already newer than 68 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 72 (ObjNum 101 snap 24 seq_num 101) dirty exists 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:216: left oid 37 (ObjNum 101 snap 24 seq_num 101) 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:221: finishing write tid 1 to smithi11731613-49 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:221: finishing write tid 2 to smithi11731613-49 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:222: finishing copy_from racing read to smithi11731613-43 2024-09-16T06:49:00.058 INFO:tasks.rados.rados.0.smithi117.stdout:221: finishing write tid 3 to smithi11731613-49 2024-09-16T06:49:00.059 INFO:tasks.rados.rados.0.smithi117.stdout:220: expect (ObjNum 73 snap 7 seq_num 73) 2024-09-16T06:49:00.063 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:00.059+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:00.164 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:00.159+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:00.182 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:00.182 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:00.183 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:49:00.183 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:49:00.183 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:00.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:49:00.265 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:00.263+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:00.285 INFO:tasks.rados.rados.0.smithi117.stdout:222: finishing copy_from to smithi11731613-43 2024-09-16T06:49:00.285 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 43 v 8 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: finishing write tid 1 to smithi11731613-34 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: finishing write tid 2 to smithi11731613-34 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:224: finishing rollback tid 0 to smithi11731613-7 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:224: finishing rollback tid 1 to smithi11731613-7 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: finishing write tid 3 to smithi11731613-34 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: finishing write tid 4 to smithi11731613-34 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: finishing write tid 5 to smithi11731613-34 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: oid 34 updating version 0 to 12 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: oid 34 updating version 12 to 13 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: oid 34 updating version 13 to 14 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: oid 34 updating version 14 to 15 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 34 v 15 (ObjNum 104 snap 24 seq_num 104) dirty exists 2024-09-16T06:49:00.286 INFO:tasks.rados.rados.0.smithi117.stdout:223: left oid 34 (ObjNum 104 snap 24 seq_num 104) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:216: done (5 left) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:220: done (4 left) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:222: done (3 left) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:223: done (2 left) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:225: read oid 3 snap -1 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:225: expect (ObjNum 99 snap 23 seq_num 99) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:226: read oid 32 snap -1 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:226: expect (ObjNum 77 snap 11 seq_num 77) 2024-09-16T06:49:00.287 INFO:tasks.rados.rados.0.smithi117.stdout:227: delete oid 30 current snap is 24 2024-09-16T06:49:00.297 INFO:tasks.rados.rados.0.smithi117.stdout:227: done (4 left) 2024-09-16T06:49:00.297 INFO:tasks.rados.rados.0.smithi117.stdout:228: setattr oid 2 current snap is 24 2024-09-16T06:49:00.299 INFO:tasks.rados.rados.0.smithi117.stdout:229: rollback oid 10 current snap is 24 2024-09-16T06:49:00.299 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 10 to 15 2024-09-16T06:49:00.299 INFO:tasks.rados.rados.0.smithi117.stdout:230: read oid 23 snap -1 2024-09-16T06:49:00.299 INFO:tasks.rados.rados.0.smithi117.stdout:230: expect (ObjNum 57 snap 3 seq_num 57) 2024-09-16T06:49:00.300 INFO:tasks.rados.rados.0.smithi117.stdout:231: delete oid 14 current snap is 24 2024-09-16T06:49:00.300 INFO:tasks.rados.rados.0.smithi117.stdout:226: expect (ObjNum 77 snap 11 seq_num 77) 2024-09-16T06:49:00.330 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:00.330 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:49:00.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:49:00.330 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:49:00.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.330 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:7 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.434977103000001, 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023918795999999999, 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:49:00.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895875+0000", 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896153+0000", 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000277688 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896194+0000", 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1916000000000003e-05 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896222+0000", 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.726e-05 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896390+0000", 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016809299999999999 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896750+0000", 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036068999999999999 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898496+0000", 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017459070000000001 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906280+0000", 2024-09-16T06:49:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077831310000000004 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919636+0000", 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013356191 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919798+0000", 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016195000000000001 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:8 3.1s0 3:825f7e25:::smithi11731613-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.434326677000001, 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031459483000000003, 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 8 2024-09-16T06:49:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896527+0000", 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896557+0000", 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9323999999999999e-05 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896578+0000", 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1038000000000001e-05 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896594+0000", 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6038000000000001e-05 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900822+0000", 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042283219999999996 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901041+0000", 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021897799999999999 2024-09-16T06:49:00.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902173+0000", 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00113226 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911105+0000", 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089313959999999994 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927760+0000", 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016655313000000001 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927989+0000", 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022873699999999999 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:9 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.434245648000001, 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.040864080999999997, 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 9 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896609+0000", 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896617+0000", 2024-09-16T06:49:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9589999999999995e-06 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896634+0000", 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6872e-05 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896647+0000", 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3621999999999999e-05 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902794+0000", 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061462820000000003 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903015+0000", 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221487 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928840+0000", 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025824686999999999 2024-09-16T06:49:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929046+0000", 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020604299999999999 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.937474+0000", 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084285460000000003 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:16 3.bs0 3:db3fccb6:::smithi11731613-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.434196037, 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024707216000000001, 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896659+0000", 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897043+0000", 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038430400000000003 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897068+0000", 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5417000000000001e-05 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897087+0000", 2024-09-16T06:49:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8218000000000001e-05 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897200+0000", 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011297499999999999 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897502+0000", 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030200600000000001 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899657+0000", 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021558250000000001 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911012+0000", 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011354401 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921200+0000", 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010188033000000001 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921367+0000", 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016735 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:17 3.bs0 3:db3fccb6:::smithi11731613-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.432969010000001, 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034791753000000002, 2024-09-16T06:49:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 17 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897885+0000", 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897900+0000", 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5515e-05 2024-09-16T06:49:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897935+0000", 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4703000000000003e-05 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897965+0000", 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9884000000000001e-05 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905007+0000", 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007042296 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905241+0000", 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023388 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014152209999999999 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912065+0000", 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054085640000000003 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932510+0000", 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020444922000000001 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932679+0000", 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168894 2024-09-16T06:49:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:18 3.bs0 3:db3fccb6:::smithi11731613-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.432810473000004, 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.041379560000000003, 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 18 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898044+0000", 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898055+0000", 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0902e-05 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898077+0000", 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2395000000000002e-05 2024-09-16T06:49:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898095+0000", 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7825e-05 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907329+0000", 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092342250000000004 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907544+0000", 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021447400000000001 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932831+0000", 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025286896999999999 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933015+0000", 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018463 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939425+0000", 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064098250000000001 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:37 3.bs0 3:d4957fab:::smithi11731613-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.425671268999999, 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029935371999999998, 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 37 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905182+0000", 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905624+0000", 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044199799999999998 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905656+0000", 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2393e-05 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905676+0000", 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0285e-05 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907630+0000", 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019531459999999998 2024-09-16T06:49:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907646+0000", 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6628000000000001e-05 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914043+0000", 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063962699999999999 2024-09-16T06:49:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914333+0000", 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029089300000000002 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916429+0000", 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002095085 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923501+0000", 2024-09-16T06:49:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007072404 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934947+0000", 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011445688000000001 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.935120+0000", 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017347200000000001 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:38 3.bs0 3:d4957fab:::smithi11731613-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.425158781999997, 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034120408999999997, 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 38 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905696+0000", 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905706+0000", 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9890000000000004e-06 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905725+0000", 2024-09-16T06:49:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9341e-05 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905739+0000", 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4336e-05 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907725+0000", 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001985849 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907741+0000", 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5401e-05 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918115+0000", 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010374431999999999 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918349+0000", 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023440900000000001 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919836+0000", 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014862639999999999 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.925093+0000", 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052577300000000004 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939675+0000", 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014581904999999999 2024-09-16T06:49:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939818+0000", 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014229600000000001 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:39 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.425102654, 2024-09-16T06:49:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039069524000000001, 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 39 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905752+0000", 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905759+0000", 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-16T06:49:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905776+0000", 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6694e-05 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905788+0000", 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2336e-05 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907813+0000", 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020251790000000002 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907828+0000", 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5094e-05 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921873+0000", 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014044235 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922092+0000", 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000219514 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940712+0000", 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018619988000000001 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940891+0000", 2024-09-16T06:49:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017858199999999999 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.944823+0000", 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039321989999999999 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:40 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:49:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.425054738999997, 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027254417, 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 40 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905800+0000", 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906496+0000", 2024-09-16T06:49:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00069684800000000002 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906524+0000", 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7344000000000001e-05 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906543+0000", 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9468000000000001e-05 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906717+0000", 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000173692 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906758+0000", 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1186000000000002e-05 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.913762+0000", 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070039339999999999 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914055+0000", 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029257000000000003 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916068+0000", 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020128730000000001 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921559+0000", 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054913260000000004 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932938+0000", 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011379031 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933056+0000", 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117785 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:41 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.424278739000002, 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027013423000000002, 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 41 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906575+0000", 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906603+0000", 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8030000000000001e-05 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906622+0000", 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9008000000000002e-05 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906638+0000", 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5469e-05 2024-09-16T06:49:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906985+0000", 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034709699999999999 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906998+0000", 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3512e-05 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917405+0000", 2024-09-16T06:49:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010406425 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917725+0000", 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032018999999999998 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919697+0000", 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019721069999999999 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924833+0000", 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051359170000000003 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933307+0000", 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084737270000000003 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933591+0000", 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028407699999999998 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:42 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.424200112999998, 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031605503, 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 42 2024-09-16T06:49:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906654+0000", 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906661+0000", 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5719999999999999e-06 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906678+0000", 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7059e-05 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906698+0000", 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9551e-05 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907075+0000", 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037732500000000002 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907087+0000", 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1745e-05 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920576+0000", 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013489655 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920843+0000", 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026655899999999998 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934288+0000", 2024-09-16T06:49:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013445297 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934560+0000", 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000272238 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.938261+0000", 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037009289999999999 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/32 ECSubWrite(tid=2, reqid=client.4511.0:19, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.423994485000001, 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018027405, 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:49:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906861+0000", 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907090+0000", 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229569 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919015+0000", 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011924924999999999 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919062+0000", 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6316000000000002e-05 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919376+0000", 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031417700000000001 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919401+0000", 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4854999999999998e-05 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924838+0000", 2024-09-16T06:49:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054377970000000003 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924889+0000", 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0445000000000002e-05 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:105 3.6s0 3:64026cd9:::smithi11731613-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.324272542999999, 2024-09-16T06:49:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023735583000000001, 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 105 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006583+0000", 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006587+0000", 2024-09-16T06:49:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6360000000000002e-06 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006598+0000", 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1151999999999999e-05 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.006608+0000", 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0036e-05 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.011103+0000", 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044949100000000004 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.011267+0000", 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000163778 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018815+0000", 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007547705 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018930+0000", 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115093 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.030319+0000", 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011389132999999999 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:108 3.6s0 3:69f2a009:::smithi11731613-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.320530763999997, 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021014327999999999, 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 108 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:49:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010325+0000", 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010324+0000", 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010330+0000", 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4829999999999996e-06 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010348+0000", 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7878000000000001e-05 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.010365+0000", 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6770999999999999e-05 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020964+0000", 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010598849 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021064+0000", 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100537 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028570+0000", 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075053480000000002 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.028742+0000", 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017269299999999999 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.031340+0000", 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002597168 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:111 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.317080713000003, 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019128665999999999, 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 111 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013775+0000", 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013774+0000", 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013781+0000", 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7320000000000001e-06 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013798+0000", 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7665000000000001e-05 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.013812+0000", 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3745e-05 2024-09-16T06:49:00.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018077+0000", 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004264561 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.018218+0000", 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014137000000000001 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.023743+0000", 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055253430000000003 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.023904+0000", 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016036100000000001 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.032904+0000", 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090001990000000004 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:113 3.1s0 3:87a9bcdd:::smithi11731613-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.313019560000001, 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017992622999999999, 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 113 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.422 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017836+0000", 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017835+0000", 2024-09-16T06:49:00.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017845+0000", 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9189999999999994e-06 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017876+0000", 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1446999999999999e-05 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017889+0000", 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2571e-05 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020765+0000", 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028765259999999999 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.020877+0000", 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111523 2024-09-16T06:49:00.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021333+0000", 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00045663300000000002 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.022097+0000", 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00076327800000000005 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035682+0000", 2024-09-16T06:49:00.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013585000999999999 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.035829+0000", 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000147367 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:114 3.1s0 3:87a9bcdd:::smithi11731613-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.312955795000001, 2024-09-16T06:49:00.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021941939000000001, 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 114 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017900+0000", 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017899+0000", 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017905+0000", 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7209999999999999e-06 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017917+0000", 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1979e-05 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.017926+0000", 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1430000000000004e-06 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021590+0000", 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036640029999999999 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.021682+0000", 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2441000000000005e-05 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036527+0000", 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014844758 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.036729+0000", 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020231099999999999 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.039842+0000", 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031126779999999998 2024-09-16T06:49:00.432 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:152 3.7s0 3:ebfb55a4:::smithi11731613-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.254448566000001, 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44514418, 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 152 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076406+0000", 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076414+0000", 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2889999999999998e-06 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076425+0000", 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0671e-05 2024-09-16T06:49:00.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076433+0000", 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0870000000000007e-06 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079120+0000", 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026864800000000002 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079194+0000", 2024-09-16T06:49:00.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4874000000000002e-05 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079523+0000", 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032904499999999998 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080194+0000", 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067098400000000001 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:00.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521344+0000", 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.441149183 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521552+0000", 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020799600000000001 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:153 3.7s0 3:ebfb55a4:::smithi11731613-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:00.438 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.254414740000001, 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44943085999999999, 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 153 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076440+0000", 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076444+0000", 2024-09-16T06:49:00.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0559999999999998e-06 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076455+0000", 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0789e-05 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076466+0000", 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0241000000000001e-05 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080491+0000", 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040256060000000001 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080588+0000", 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6849999999999996e-05 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522277+0000", 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44168930299999998 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522519+0000", 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241246 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.525872+0000", 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033535660000000001 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:00.443 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:00.444 INFO:tasks.rados.rados.0.smithi117.stdout:229: finishing rollback tid 0 to smithi11731613-10 2024-09-16T06:49:00.444 INFO:tasks.rados.rados.0.smithi117.stdout:225: expect (ObjNum 99 snap 23 seq_num 99) 2024-09-16T06:49:00.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:49:00.446 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:00.444+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:221: finishing write tid 4 to smithi11731613-49 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 2 v 106 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:229: finishing rollback tid 1 to smithi11731613-10 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:229: finishing rollback tid 2 to smithi11731613-10 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 135 (ObjNum 78 snap 11 seq_num 78) dirty exists 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:221: finishing write tid 5 to smithi11731613-49 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:221: oid 49 updating version 0 to 76 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:221: oid 49 updating version 76 to 77 2024-09-16T06:49:00.450 INFO:tasks.rados.rados.0.smithi117.stdout:221: oid 49 version 77 is already newer than 74 2024-09-16T06:49:00.451 INFO:tasks.rados.rados.0.smithi117.stdout:221: oid 49 version 77 is already newer than 75 2024-09-16T06:49:00.451 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 49 v 77 (ObjNum 103 snap 24 seq_num 103) dirty exists 2024-09-16T06:49:00.451 INFO:tasks.rados.rados.0.smithi117.stdout:221: left oid 49 (ObjNum 103 snap 24 seq_num 103) 2024-09-16T06:49:00.451 INFO:tasks.rados.rados.0.smithi117.stdout:230: expect (ObjNum 57 snap 3 seq_num 57) 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:224: finishing rollback tid 2 to smithi11731613-7 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 7 v 94 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:221: done (7 left) 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:224: done (6 left) 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:225: done (5 left) 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:226: done (4 left) 2024-09-16T06:49:00.491 INFO:tasks.rados.rados.0.smithi117.stdout:228: done (3 left) 2024-09-16T06:49:00.492 INFO:tasks.rados.rados.0.smithi117.stdout:229: done (2 left) 2024-09-16T06:49:00.492 INFO:tasks.rados.rados.0.smithi117.stdout:230: done (1 left) 2024-09-16T06:49:00.492 INFO:tasks.rados.rados.0.smithi117.stdout:231: done (0 left) 2024-09-16T06:49:00.492 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 37 current snap is 24 2024-09-16T06:49:00.492 INFO:tasks.rados.rados.0.smithi117.stdout:232: seq_num 107 ranges {5472256=638976,6111232=458752,6569984=704512} 2024-09-16T06:49:00.545 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:00.543+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:00.567 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:00.567 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:00.567 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:00.568 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:00.572 INFO:tasks.rados.rados.0.smithi117.stdout:232: writing smithi11731613-37 from 5472256 to 6111232 tid 1 2024-09-16T06:49:00.575 INFO:tasks.rados.rados.0.smithi117.stdout:232: writing smithi11731613-37 from 6111232 to 6569984 tid 2 2024-09-16T06:49:00.581 INFO:tasks.rados.rados.0.smithi117.stdout:232: writing smithi11731613-37 from 6569984 to 7274496 tid 3 2024-09-16T06:49:00.581 INFO:tasks.rados.rados.0.smithi117.stdout:233: rollback oid 3 current snap is 24 2024-09-16T06:49:00.581 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 3 to 14 2024-09-16T06:49:00.582 INFO:tasks.rados.rados.0.smithi117.stdout:234: delete oid 39 current snap is 24 2024-09-16T06:49:00.582 INFO:tasks.rados.rados.0.smithi117.stdout:232: finishing write tid 1 to smithi11731613-37 2024-09-16T06:49:00.583 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:49:00.588 INFO:tasks.rados.rados.0.smithi117.stdout:234: done (2 left) 2024-09-16T06:49:00.588 INFO:tasks.rados.rados.0.smithi117.stdout:235: rmattr oid 15 current snap is 24 2024-09-16T06:49:00.588 INFO:tasks.rados.rados.0.smithi117.stdout:236: delete oid 1 current snap is 24 2024-09-16T06:49:00.592 INFO:tasks.rados.rados.0.smithi117.stdout:233: finishing rollback tid 0 to smithi11731613-3 2024-09-16T06:49:00.592 INFO:tasks.rados.rados.0.smithi117.stdout:233: finishing rollback tid 1 to smithi11731613-3 2024-09-16T06:49:00.593 INFO:tasks.rados.rados.0.smithi117.stdout:233: finishing rollback tid 2 to smithi11731613-3 2024-09-16T06:49:00.594 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 40 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-16T06:49:00.594 INFO:tasks.rados.rados.0.smithi117.stdout:232: finishing write tid 2 to smithi11731613-37 2024-09-16T06:49:00.594 INFO:tasks.rados.rados.0.smithi117.stdout:232: finishing write tid 3 to smithi11731613-37 2024-09-16T06:49:00.596 INFO:tasks.rados.rados.0.smithi117.stdout:232: finishing write tid 4 to smithi11731613-37 2024-09-16T06:49:00.598 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 15 v 110 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:00.599 INFO:tasks.rados.rados.0.smithi117.stdout:233: done (3 left) 2024-09-16T06:49:00.599 INFO:tasks.rados.rados.0.smithi117.stdout:235: done (2 left) 2024-09-16T06:49:00.599 INFO:tasks.rados.rados.0.smithi117.stdout:236: done (1 left) 2024-09-16T06:49:00.599 INFO:tasks.rados.rados.0.smithi117.stdout:237: delete oid 3 current snap is 24 2024-09-16T06:49:00.602 INFO:tasks.rados.rados.0.smithi117.stdout:232: finishing write tid 5 to smithi11731613-37 2024-09-16T06:49:00.602 INFO:tasks.rados.rados.0.smithi117.stdout:232: oid 37 updating version 0 to 80 2024-09-16T06:49:00.602 INFO:tasks.rados.rados.0.smithi117.stdout:232: oid 37 version 80 is already newer than 79 2024-09-16T06:49:00.602 INFO:tasks.rados.rados.0.smithi117.stdout:232: oid 37 updating version 80 to 82 2024-09-16T06:49:00.602 INFO:tasks.rados.rados.0.smithi117.stdout:232: oid 37 version 82 is already newer than 81 2024-09-16T06:49:00.603 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 82 (ObjNum 107 snap 24 seq_num 107) dirty exists 2024-09-16T06:49:00.603 INFO:tasks.rados.rados.0.smithi117.stdout:232: left oid 37 (ObjNum 107 snap 24 seq_num 107) 2024-09-16T06:49:00.604 INFO:tasks.rados.rados.0.smithi117.stdout:232: done (1 left) 2024-09-16T06:49:00.604 INFO:tasks.rados.rados.0.smithi117.stdout:237: done (0 left) 2024-09-16T06:49:00.604 INFO:tasks.rados.rados.0.smithi117.stdout:238: delete oid 32 current snap is 24 2024-09-16T06:49:00.610 INFO:tasks.rados.rados.0.smithi117.stdout:238: done (0 left) 2024-09-16T06:49:00.610 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 44 current snap is 24 2024-09-16T06:49:00.610 INFO:tasks.rados.rados.0.smithi117.stdout:239: seq_num 112 ranges {1802240=720896,2523136=802816,3325952=360448} 2024-09-16T06:49:00.639 INFO:tasks.rados.rados.0.smithi117.stdout:239: writing smithi11731613-44 from 1802240 to 2523136 tid 1 2024-09-16T06:49:00.646 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:00.646+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:00.648 INFO:tasks.rados.rados.0.smithi117.stdout:239: writing smithi11731613-44 from 2523136 to 3325952 tid 2 2024-09-16T06:49:00.652 INFO:tasks.rados.rados.0.smithi117.stdout:239: writing smithi11731613-44 from 3325952 to 3686400 tid 3 2024-09-16T06:49:00.652 INFO:tasks.rados.rados.0.smithi117.stdout:240: read oid 32 snap -1 2024-09-16T06:49:00.652 INFO:tasks.rados.rados.0.smithi117.stdout:240: expect deleted 2024-09-16T06:49:00.653 INFO:tasks.rados.rados.0.smithi117.stdout:241: snap_remove snap 11 2024-09-16T06:49:00.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:00.747 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:00.747+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:00.752 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:00.752 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:00.753 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:00.753 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:00.753 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:00.768 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:241: done (2 left) 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:242: snap_create 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: finishing write tid 1 to smithi11731613-44 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: finishing write tid 2 to smithi11731613-44 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: finishing write tid 3 to smithi11731613-44 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: finishing write tid 4 to smithi11731613-44 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: finishing write tid 5 to smithi11731613-44 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: oid 44 updating version 0 to 94 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: oid 44 updating version 94 to 95 2024-09-16T06:49:00.843 INFO:tasks.rados.rados.0.smithi117.stdout:239: oid 44 version 95 is already newer than 93 2024-09-16T06:49:00.844 INFO:tasks.rados.rados.0.smithi117.stdout:239: oid 44 version 95 is already newer than 92 2024-09-16T06:49:00.844 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 95 (ObjNum 112 snap 24 seq_num 112) dirty exists 2024-09-16T06:49:00.844 INFO:tasks.rados.rados.0.smithi117.stdout:239: left oid 44 (ObjNum 112 snap 24 seq_num 112) 2024-09-16T06:49:00.848 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:00.846+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:49:00.876 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:00.876 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:00.901 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:00.902 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:00.905 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=5, reqid=client.4511.0:59, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.963485439000003, 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063718670000000005, 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941153+0000", 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941187+0000", 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3837000000000001e-05 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002113+0000", 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060926368000000002 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002134+0000", 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-05 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002230+0000", 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5789000000000001e-05 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002246+0000", 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5676e-05 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004821+0000", 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002575262 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004873+0000", 2024-09-16T06:49:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2095000000000002e-05 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=158, reqid=client.4511.0:77, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.929088401000001, 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051524863999999997, 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975550+0000", 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975589+0000", 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9523000000000001e-05 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021558+0000", 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045968459000000003 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021574+0000", 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6399999999999999e-05 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021656+0000", 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1742000000000004e-05 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021673+0000", 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7343e-05 2024-09-16T06:49:00.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027012+0000", 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053383889999999998 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027076+0000", 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4325999999999994e-05 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.9s1 35/32 ECSubRead(tid=11, to_read={3:92c79891:::smithi11731613-34:head=0,4096,0}, subchunks={3:92c79891:::smithi11731613-34:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.893276884000002, 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034091585000000001, 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011362+0000", 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011374+0000", 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2490000000000001e-05 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044980+0000", 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033605818000000003 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044996+0000", 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5996999999999999e-05 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045095+0000", 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9290000000000007e-05 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045454+0000", 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000359038 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.859048438999999, 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623068000000002, 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045590+0000", 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045617+0000", 2024-09-16T06:49:00.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.724e-05 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073137+0000", 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027519538999999999 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073150+0000", 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3046e-05 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073218+0000", 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7879999999999994e-05 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073228+0000", 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074167+0000", 2024-09-16T06:49:00.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093834699999999997 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074214+0000", 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7722000000000002e-05 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.0s3 35/32 ECSubRead(tid=19, to_read={3:043b7e8c:::smithi11731613-16:head=0,65536,0}, subchunks={3:043b7e8c:::smithi11731613-16:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.822894673, 2024-09-16T06:49:00.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.14017474999999999, 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081744+0000", 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081755+0000", 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0812000000000001e-05 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221448+0000", 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.13969311000000001 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221465+0000", 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7030000000000001e-05 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221629+0000", 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163549 2024-09-16T06:49:00.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221920+0000", 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000291181 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.5s3 41/37 ECSubRead(tid=7, to_read={3:af79444c:::smithi11731613-29:head=0,118784,0}, subchunks={3:af79444c:::smithi11731613-29:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:00.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.940870867000001, 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29065317299999999, 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:00.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963768+0000", 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963777+0000", 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2019999999999993e-06 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254192+0000", 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.290415064 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254204+0000", 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1430000000000001e-05 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254309+0000", 2024-09-16T06:49:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104763 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254422+0000", 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011342500000000001 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.048111647999999, 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017061956, 2024-09-16T06:49:00.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856527+0000", 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856580+0000", 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2326e-05 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856691+0000", 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011141600000000001 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856715+0000", 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3710000000000002e-05 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856826+0000", 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011109599999999999 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856852+0000", 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5686000000000001e-05 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873537+0000", 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016685114000000001 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873590+0000", 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3242000000000003e-05 2024-09-16T06:49:00.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 59/32 ECSubWrite(tid=54, reqid=client.4511.0:469, at_version=59'36, trim_to=0'0, roll_forward_to=57'34))", 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.064042457999999, 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91193373600000005, 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840907+0000", 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000310045 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749945+0000", 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.90903863699999998 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749964+0000", 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8073000000000001e-05 2024-09-16T06:49:00.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750059+0000", 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5636000000000004e-05 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750079+0000", 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9893000000000001e-05 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752481+0000", 2024-09-16T06:49:00.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024015989999999999 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752531+0000", 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0263000000000003e-05 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.941 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=90, reqid=osd.1.0:90, at_version=61'54, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.087715175, 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018170465, 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816986+0000", 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1818999999999999e-05 2024-09-16T06:49:00.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817088+0000", 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102377 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817111+0000", 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2946e-05 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817267+0000", 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015629899999999999 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817294+0000", 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6412999999999999e-05 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835044+0000", 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017750359 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835095+0000", 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1004999999999997e-05 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.084671234, 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175777999999999, 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819967+0000", 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820019+0000", 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2649999999999999e-05 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820184+0000", 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016481499999999999 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820207+0000", 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.245e-05 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820328+0000", 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120942 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820352+0000", 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4732999999999999e-05 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835111+0000", 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014758378000000001 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835144+0000", 2024-09-16T06:49:00.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3547000000000003e-05 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0727022979999998, 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016104653999999999, 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831994+0000", 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7025999999999999e-05 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832116+0000", 2024-09-16T06:49:00.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012219299999999999 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832140+0000", 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4023e-05 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832493+0000", 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035320700000000002 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832518+0000", 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5147000000000002e-05 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847982+0000", 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015463289 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848042+0000", 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0463e-05 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.es4 71/37 ECSubWrite(tid=22, reqid=osd.8.0:22, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0702084269999999, 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019478348999999999, 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:00.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834430+0000", 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834474+0000", 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3739999999999998e-05 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834597+0000", 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122753 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834618+0000", 2024-09-16T06:49:00.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1313000000000001e-05 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834923+0000", 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030448399999999999 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834948+0000", 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5293e-05 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853837+0000", 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018889153999999998 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853910+0000", 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2368999999999998e-05 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.960 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=191, reqid=osd.11.0:191, at_version=71'82, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0687577410000002, 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175754, 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835880+0000", 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835936+0000", 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6512999999999999e-05 2024-09-16T06:49:00.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836143+0000", 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020638500000000001 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836164+0000", 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1169999999999999e-05 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836644+0000", 2024-09-16T06:49:00.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047988000000000002 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836664+0000", 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0616999999999999e-05 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.851001+0000", 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014336389999999999 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.851058+0000", 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6969000000000003e-05 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=144, reqid=client.4511.0:542, at_version=71'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0636939949999999, 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016044704, 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840944+0000", 2024-09-16T06:49:00.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841000+0000", 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5287000000000001e-05 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841092+0000", 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2009000000000006e-05 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841121+0000", 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.885e-05 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841388+0000", 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026696100000000002 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841425+0000", 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7626999999999997e-05 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856931+0000", 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015505755 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856990+0000", 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9401999999999998e-05 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0631159019999998, 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016485414, 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841522+0000", 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841612+0000", 2024-09-16T06:49:00.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0575999999999995e-05 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841764+0000", 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015128200000000001 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841786+0000", 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1759999999999998e-05 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848074+0000", 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062882529999999997 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848099+0000", 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5510000000000001e-05 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857929+0000", 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098292880000000003 2024-09-16T06:49:00.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858009+0000", 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0599000000000005e-05 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=146, reqid=client.4511.0:543, at_version=71'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.062348235, 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018827678, 2024-09-16T06:49:00.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842289+0000", 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842343+0000", 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842426+0000", 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3009000000000004e-05 2024-09-16T06:49:00.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842448+0000", 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1860000000000001e-05 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850355+0000", 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079069680000000003 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850377+0000", 2024-09-16T06:49:00.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2167e-05 2024-09-16T06:49:00.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861052+0000", 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010674678 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861119+0000", 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7303000000000003e-05 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=147, reqid=client.4511.0:544, at_version=71'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.0587119779999998, 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015229647000000001, 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:00.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845926+0000", 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845972+0000", 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5745999999999999e-05 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846062+0000", 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9909999999999998e-05 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846133+0000", 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1109e-05 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853380+0000", 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072475899999999999 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853426+0000", 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5456e-05 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861126+0000", 2024-09-16T06:49:00.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077004730000000002 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861157+0000", 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1130000000000002e-05 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=132, reqid=osd.1.0:132, at_version=72'83, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.0613048439999999, 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015387283999999999, 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843334+0000", 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844171+0000", 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083714800000000002 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845132+0000", 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096062600000000001 2024-09-16T06:49:00.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845482+0000", 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035013699999999999 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845634+0000", 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015229600000000001 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845661+0000", 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858652+0000", 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012991745000000001 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858722+0000", 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9822999999999999e-05 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s6 72/37 ECSubWrite(tid=133, reqid=osd.1.0:133, at_version=72'36, trim_to=0'0, roll_forward_to=71'34))", 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.0604554900000001, 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017512679999999999, 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:00.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844183+0000", 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844237+0000", 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4425999999999997e-05 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845784+0000", 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015470320000000001 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845809+0000", 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.455e-05 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845934+0000", 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012515900000000001 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845958+0000", 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3879999999999998e-05 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861640+0000", 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01568172 2024-09-16T06:49:00.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861697+0000", 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7173000000000003e-05 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=134, reqid=osd.1.0:134, at_version=72'85, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.059720692, 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017033627999999999, 2024-09-16T06:49:00.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:00.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844918+0000", 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844980+0000", 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1920000000000003e-05 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846382+0000", 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014028630000000001 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846403+0000", 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0143e-05 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848953+0000", 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002550667 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848977+0000", 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3337e-05 2024-09-16T06:49:00.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861911+0000", 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012934783 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861953+0000", 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1192000000000001e-05 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:00.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:01.000 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:01.000 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:01.001 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:01.001 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:01.001 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:49:01.002 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:00.999+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:01.002 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:01.002 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:01.002 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:49:01.003 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:49:01.028 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:01.056 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:01.056 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:01.085 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:01.085 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:01.101 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:01.099+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:01.116 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:01.147 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:01.172 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:01.173 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:01.178 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:01.186 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:49:01.202 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:01.200+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:01.207 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:01.239 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:01.240 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:01.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:01.268+0000 7ff52b7fe640 0 -- 172.21.15.100:0/46343064 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7ff52c222980 msgr2=0x7ff52c224d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:01.277 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:01.303 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:01.300+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:01.311 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:01.311 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:01.348 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:01.348 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:01.364 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:49:01.403 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:01.402+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:01.487 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:01.487 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:01.501 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:49:01.504 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:01.502+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:01.604 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:01.603+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:01.618 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:01.618 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:01.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:49:01.724 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:01.723+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:01.800 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:01.800 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:01.806 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:01.808+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:01.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:49:01.817 INFO:tasks.rados.rados.0.smithi117.stdout:239: done (2 left) 2024-09-16T06:49:01.817 INFO:tasks.rados.rados.0.smithi117.stdout:240: done (1 left) 2024-09-16T06:49:01.817 INFO:tasks.rados.rados.0.smithi117.stdout:242: done (0 left) 2024-09-16T06:49:01.817 INFO:tasks.rados.rados.0.smithi117.stdout:243: read oid 1 snap 22 2024-09-16T06:49:01.817 INFO:tasks.rados.rados.0.smithi117.stdout:243: expect (ObjNum 55 snap 2 seq_num 55) 2024-09-16T06:49:01.817 INFO:tasks.rados.rados.0.smithi117.stdout:244: snap_create 2024-09-16T06:49:01.832 INFO:tasks.rados.rados.0.smithi117.stdout:243: expect (ObjNum 55 snap 2 seq_num 55) 2024-09-16T06:49:01.872 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:49:01.919 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:01.920+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:01.972 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:01.972 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:01.986 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:49:02.008 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:02.008+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:02.103 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.104 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.109 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:02.108+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:02.118 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:49:02.210 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:02.207+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:02.233 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.234 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:49:02.310 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:02.307+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:02.366 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.367 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:49:02.411 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:02.407+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:02.510 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.510 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.512 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:02.512+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:02.524 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:49:02.613 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:02.612+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:02.645 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.645 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.659 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:49:02.714 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:02.715+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:02.780 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.780 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.794 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:49:02.814 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:02.811+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:02.818 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:49:02.820 INFO:tasks.rados.rados.0.smithi117.stdout:243: done (1 left) 2024-09-16T06:49:02.820 INFO:tasks.rados.rados.0.smithi117.stdout:244: done (0 left) 2024-09-16T06:49:02.820 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 34 current snap is 26 2024-09-16T06:49:02.820 INFO:tasks.rados.rados.0.smithi117.stdout:245: seq_num 113 ranges {2310144=737280,3047424=557056,3604480=606208} 2024-09-16T06:49:02.841 INFO:tasks.rados.rados.0.smithi117.stdout:245: writing smithi11731613-34 from 2310144 to 3047424 tid 1 2024-09-16T06:49:02.845 INFO:tasks.rados.rados.0.smithi117.stdout:245: writing smithi11731613-34 from 3047424 to 3604480 tid 2 2024-09-16T06:49:02.850 INFO:tasks.rados.rados.0.smithi117.stdout:245: writing smithi11731613-34 from 3604480 to 4210688 tid 3 2024-09-16T06:49:02.850 INFO:tasks.rados.rados.0.smithi117.stdout:246: rollback oid 10 current snap is 26 2024-09-16T06:49:02.850 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 10 to 23 2024-09-16T06:49:02.851 INFO:tasks.rados.rados.0.smithi117.stdout:247: read oid 12 snap 23 2024-09-16T06:49:02.851 INFO:tasks.rados.rados.0.smithi117.stdout:247: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:02.851 INFO:tasks.rados.rados.0.smithi117.stdout:248: setattr oid 39 current snap is 26 2024-09-16T06:49:02.852 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 47 current snap is 26 2024-09-16T06:49:02.853 INFO:tasks.rados.rados.0.smithi117.stdout:249: seq_num 114 ranges {3031040=753664,3784704=688128,4472832=475136} 2024-09-16T06:49:02.876 INFO:tasks.rados.rados.0.smithi117.stdout:249: writing smithi11731613-47 from 3031040 to 3784704 tid 1 2024-09-16T06:49:02.882 INFO:tasks.rados.rados.0.smithi117.stdout:249: writing smithi11731613-47 from 3784704 to 4472832 tid 2 2024-09-16T06:49:02.885 INFO:tasks.rados.rados.0.smithi117.stdout:249: writing smithi11731613-47 from 4472832 to 4947968 tid 3 2024-09-16T06:49:02.886 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 18 current snap is 26 2024-09-16T06:49:02.886 INFO:tasks.rados.rados.0.smithi117.stdout:250: seq_num 115 ranges {1802240=770048,2572288=704512,3276800=458752} 2024-09-16T06:49:02.902 INFO:tasks.rados.rados.0.smithi117.stdout:250: writing smithi11731613-18 from 1802240 to 2572288 tid 1 2024-09-16T06:49:02.907 INFO:tasks.rados.rados.0.smithi117.stdout:250: writing smithi11731613-18 from 2572288 to 3276800 tid 2 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:250: writing smithi11731613-18 from 3276800 to 3735552 tid 3 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:251: copy_from oid 25 from oid 6 current snap is 26 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:245: finishing write tid 1 to smithi11731613-34 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:252: read oid 2 snap -1 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:252: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:253: setattr oid 19 current snap is 26 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:245: finishing write tid 2 to smithi11731613-34 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:245: finishing write tid 3 to smithi11731613-34 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:245: finishing write tid 4 to smithi11731613-34 2024-09-16T06:49:02.911 INFO:tasks.rados.rados.0.smithi117.stdout:247: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:02.913 INFO:tasks.rados.rados.0.smithi117.stdout:245: finishing write tid 5 to smithi11731613-34 2024-09-16T06:49:02.913 INFO:tasks.rados.rados.0.smithi117.stdout:245: oid 34 updating version 0 to 22 2024-09-16T06:49:02.913 INFO:tasks.rados.rados.0.smithi117.stdout:245: oid 34 version 22 is already newer than 19 2024-09-16T06:49:02.913 INFO:tasks.rados.rados.0.smithi117.stdout:245: oid 34 version 22 is already newer than 21 2024-09-16T06:49:02.913 INFO:tasks.rados.rados.0.smithi117.stdout:245: oid 34 version 22 is already newer than 20 2024-09-16T06:49:02.913 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 34 v 22 (ObjNum 113 snap 26 seq_num 113) dirty exists 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:245: left oid 34 (ObjNum 113 snap 26 seq_num 113) 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 39 v 119 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:246: finishing rollback tid 0 to smithi11731613-10 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:246: finishing rollback tid 1 to smithi11731613-10 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:246: finishing rollback tid 2 to smithi11731613-10 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 147 (ObjNum 78 snap 11 seq_num 78) dirty exists 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: finishing write tid 1 to smithi11731613-47 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: finishing write tid 2 to smithi11731613-47 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: finishing write tid 3 to smithi11731613-47 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: finishing write tid 4 to smithi11731613-47 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: finishing write tid 5 to smithi11731613-47 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: oid 47 updating version 0 to 104 2024-09-16T06:49:02.914 INFO:tasks.rados.rados.0.smithi117.stdout:249: oid 47 version 104 is already newer than 102 2024-09-16T06:49:02.915 INFO:tasks.rados.rados.0.smithi117.stdout:249: oid 47 version 104 is already newer than 101 2024-09-16T06:49:02.915 INFO:tasks.rados.rados.0.smithi117.stdout:249: oid 47 version 104 is already newer than 103 2024-09-16T06:49:02.915 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 47 v 104 (ObjNum 114 snap 26 seq_num 114) dirty exists 2024-09-16T06:49:02.915 INFO:tasks.rados.rados.0.smithi117.stdout:249: left oid 47 (ObjNum 114 snap 26 seq_num 114) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:245: done (8 left) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:246: done (7 left) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:247: done (6 left) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:248: done (5 left) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:249: done (4 left) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:254: read oid 8 snap -1 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:254: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:49:02.916 INFO:tasks.rados.rados.0.smithi117.stdout:255: rollback oid 4 current snap is 26 2024-09-16T06:49:02.917 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 4 to 14 2024-09-16T06:49:02.917 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 37 current snap is 26 2024-09-16T06:49:02.917 INFO:tasks.rados.rados.0.smithi117.stdout:256: seq_num 116 ranges {7274496=786432,8060928=458752,8519680=442368,8962048=262144} 2024-09-16T06:49:02.917 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:02.911+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:02.961 INFO:tasks.rados.rados.0.smithi117.stdout:256: writing smithi11731613-37 from 7274496 to 8060928 tid 1 2024-09-16T06:49:02.963 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:02.963 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:02.964 INFO:tasks.rados.rados.0.smithi117.stdout:256: writing smithi11731613-37 from 8060928 to 8519680 tid 2 2024-09-16T06:49:02.966 INFO:tasks.rados.rados.0.smithi117.stdout:256: writing smithi11731613-37 from 8519680 to 8962048 tid 3 2024-09-16T06:49:02.968 INFO:tasks.rados.rados.0.smithi117.stdout:256: writing smithi11731613-37 from 8962048 to 9224192 tid 4 2024-09-16T06:49:02.968 INFO:tasks.rados.rados.0.smithi117.stdout:257: snap_create 2024-09-16T06:49:02.968 INFO:tasks.rados.rados.0.smithi117.stdout:250: finishing write tid 1 to smithi11731613-18 2024-09-16T06:49:02.968 INFO:tasks.rados.rados.0.smithi117.stdout:250: finishing write tid 2 to smithi11731613-18 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: finishing write tid 3 to smithi11731613-18 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: finishing write tid 4 to smithi11731613-18 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: finishing write tid 5 to smithi11731613-18 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: oid 18 updating version 0 to 23 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: oid 18 updating version 23 to 26 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: oid 18 version 26 is already newer than 25 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: oid 18 version 26 is already newer than 24 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 18 v 26 (ObjNum 115 snap 26 seq_num 115) dirty exists 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:250: left oid 18 (ObjNum 115 snap 26 seq_num 115) 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:255: finishing rollback tid 0 to smithi11731613-4 2024-09-16T06:49:02.969 INFO:tasks.rados.rados.0.smithi117.stdout:255: finishing rollback tid 1 to smithi11731613-4 2024-09-16T06:49:02.970 INFO:tasks.rados.rados.0.smithi117.stdout:255: finishing rollback tid 2 to smithi11731613-4 2024-09-16T06:49:02.970 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 4 v 93 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T06:49:02.970 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 19 v 92 (ObjNum 94 snap 16 seq_num 94) dirty exists 2024-09-16T06:49:02.970 INFO:tasks.rados.rados.0.smithi117.stdout:254: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:49:02.978 INFO:tasks.rados.rados.0.smithi117.stdout:251: finishing copy_from racing read to smithi11731613-25 2024-09-16T06:49:02.978 INFO:tasks.rados.rados.0.smithi117.stdout:251: finishing copy_from to smithi11731613-25 2024-09-16T06:49:02.978 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 25 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:02.979 INFO:tasks.rados.rados.0.smithi117.stdout:256: finishing write tid 1 to smithi11731613-37 2024-09-16T06:49:02.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:49:02.981 INFO:tasks.rados.rados.0.smithi117.stdout:256: finishing write tid 2 to smithi11731613-37 2024-09-16T06:49:02.981 INFO:tasks.rados.rados.0.smithi117.stdout:256: finishing write tid 3 to smithi11731613-37 2024-09-16T06:49:02.984 INFO:tasks.rados.rados.0.smithi117.stdout:256: finishing write tid 4 to smithi11731613-37 2024-09-16T06:49:03.020 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:03.015+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:03.100 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:03.100 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:03.114 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:49:03.118 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:03.118+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:03.219 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:03.220+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:03.238 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:03.238 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:03.252 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:49:03.274 INFO:tasks.rados.rados.0.smithi117.stdout:256: finishing write tid 5 to smithi11731613-37 2024-09-16T06:49:03.285 INFO:tasks.rados.rados.0.smithi117.stdout:256: finishing write tid 6 to smithi11731613-37 2024-09-16T06:49:03.285 INFO:tasks.rados.rados.0.smithi117.stdout:256: oid 37 updating version 0 to 92 2024-09-16T06:49:03.285 INFO:tasks.rados.rados.0.smithi117.stdout:256: oid 37 updating version 92 to 94 2024-09-16T06:49:03.285 INFO:tasks.rados.rados.0.smithi117.stdout:256: oid 37 version 94 is already newer than 93 2024-09-16T06:49:03.285 INFO:tasks.rados.rados.0.smithi117.stdout:256: oid 37 version 94 is already newer than 91 2024-09-16T06:49:03.285 INFO:tasks.rados.rados.0.smithi117.stdout:256: oid 37 version 94 is already newer than 90 2024-09-16T06:49:03.286 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 37 v 94 (ObjNum 116 snap 26 seq_num 116) dirty exists 2024-09-16T06:49:03.286 INFO:tasks.rados.rados.0.smithi117.stdout:256: left oid 37 (ObjNum 116 snap 26 seq_num 116) 2024-09-16T06:49:03.288 INFO:tasks.rados.rados.0.smithi117.stdout:252: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:03.320 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:03.319+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:03.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:03.378 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:03.378 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:03.392 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:49:03.420 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:03.419+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:03.504 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:03.504 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:03.521 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:03.519+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:03.528 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:03.528 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:03.555 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:03.555 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:03.556 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:03.556 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:03.571 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:49:03.582 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:03.582 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:03.609 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:03.609 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:03.622 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:03.619+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:03.634 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:49:03.635 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:49:03.663 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:03.664 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:03.693 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:03.693 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:03.694 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:03.694 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:03.708 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:49:03.722 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:03.722 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:03.722 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:03.720+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:03.753 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:03.753 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:03.783 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:03.813 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:03.824 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:03.822+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:03.832 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:03.832 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:03.841 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:03.841 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:03.846 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:250: done (7 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:251: done (6 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:252: done (5 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:253: done (4 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:254: done (3 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:255: done (2 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:256: done (1 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:257: done (0 left) 2024-09-16T06:49:03.863 INFO:tasks.rados.rados.0.smithi117.stdout:258: rmattr oid 27 current snap is 27 2024-09-16T06:49:03.864 INFO:tasks.rados.rados.0.smithi117.stdout:258: done (0 left) 2024-09-16T06:49:03.864 INFO:tasks.rados.rados.0.smithi117.stdout:259: rollback oid 46 current snap is 27 2024-09-16T06:49:03.864 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 46 to 22 2024-09-16T06:49:03.864 INFO:tasks.rados.rados.0.smithi117.stdout:260: setattr oid 26 current snap is 27 2024-09-16T06:49:03.864 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15] out_osds: [13] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:49:03.864 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:49:03.864 INFO:tasks.thrashosds.thrasher:Adding osd 13 2024-09-16T06:49:03.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 13 2024-09-16T06:49:03.865 INFO:tasks.rados.rados.0.smithi117.stdout:259: finishing rollback tid 1 to smithi11731613-46 2024-09-16T06:49:03.865 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 46 v 0 (ObjNum 129 snap 0 seq_num 0) dirty dne 2024-09-16T06:49:03.865 INFO:tasks.rados.rados.0.smithi117.stdout:259: done (1 left) 2024-09-16T06:49:03.866 INFO:tasks.rados.rados.0.smithi117.stdout:261: rmattr oid 5 current snap is 27 2024-09-16T06:49:03.866 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 34 current snap is 27 2024-09-16T06:49:03.866 INFO:tasks.rados.rados.0.smithi117.stdout:262: seq_num 117 ranges {4210688=425984,4636672=606208,5242880=540672,5783552=409600} 2024-09-16T06:49:03.877 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:03.878 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:03.892 INFO:tasks.rados.rados.0.smithi117.stdout:262: writing smithi11731613-34 from 4210688 to 4636672 tid 1 2024-09-16T06:49:03.896 INFO:tasks.rados.rados.0.smithi117.stdout:262: writing smithi11731613-34 from 4636672 to 5242880 tid 2 2024-09-16T06:49:03.900 INFO:tasks.rados.rados.0.smithi117.stdout:262: writing smithi11731613-34 from 5242880 to 5783552 tid 3 2024-09-16T06:49:03.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:03.900+0000 7fe04cc6a640 0 -- 172.21.15.100:0/2595303155 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7fe04821eed0 msgr2=0x7fe0482212c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-16T06:49:03.903 INFO:tasks.rados.rados.0.smithi117.stdout:262: writing smithi11731613-34 from 5783552 to 6193152 tid 4 2024-09-16T06:49:03.903 INFO:tasks.rados.rados.0.smithi117.stdout:263: delete oid 31 current snap is 27 2024-09-16T06:49:03.903 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T06:49:03.903 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 26 v 34 (ObjNum 65 snap 0 seq_num 1678103472) dirty exists 2024-09-16T06:49:03.903 INFO:tasks.rados.rados.0.smithi117.stdout:262: finishing write tid 1 to smithi11731613-34 2024-09-16T06:49:03.903 INFO:tasks.rados.rados.0.smithi117.stdout:262: finishing write tid 2 to smithi11731613-34 2024-09-16T06:49:03.907 INFO:tasks.rados.rados.0.smithi117.stdout:262: finishing write tid 3 to smithi11731613-34 2024-09-16T06:49:03.910 INFO:tasks.rados.rados.0.smithi117.stdout:262: finishing write tid 4 to smithi11731613-34 2024-09-16T06:49:03.910 INFO:tasks.rados.rados.0.smithi117.stdout:260: done (3 left) 2024-09-16T06:49:03.911 INFO:tasks.rados.rados.0.smithi117.stdout:261: done (2 left) 2024-09-16T06:49:03.911 INFO:tasks.rados.rados.0.smithi117.stdout:263: done (1 left) 2024-09-16T06:49:03.911 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 1 current snap is 27 2024-09-16T06:49:03.911 INFO:tasks.rados.rados.0.smithi117.stdout:264: seq_num 119 ranges {0=458752,458752=507904,966656=507904,1474560=458752,1933312=81920} 2024-09-16T06:49:03.913 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:03.913 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:03.914 INFO:tasks.rados.rados.0.smithi117.stdout:264: writing smithi11731613-1 from 0 to 458752 tid 1 2024-09-16T06:49:03.918 INFO:tasks.rados.rados.0.smithi117.stdout:264: writing smithi11731613-1 from 458752 to 966656 tid 2 2024-09-16T06:49:03.922 INFO:tasks.rados.rados.0.smithi117.stdout:264: writing smithi11731613-1 from 966656 to 1474560 tid 3 2024-09-16T06:49:03.926 INFO:tasks.rados.rados.0.smithi117.stdout:264: writing smithi11731613-1 from 1474560 to 1933312 tid 4 2024-09-16T06:49:03.926 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:03.919+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:03.926 INFO:tasks.rados.rados.0.smithi117.stdout:264: writing smithi11731613-1 from 1933312 to 2015232 tid 5 2024-09-16T06:49:03.927 INFO:tasks.rados.rados.0.smithi117.stdout:265: snap_create 2024-09-16T06:49:03.927 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 1 to smithi11731613-1 2024-09-16T06:49:03.927 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 2 to smithi11731613-1 2024-09-16T06:49:03.927 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 3 to smithi11731613-1 2024-09-16T06:49:03.930 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 4 to smithi11731613-1 2024-09-16T06:49:03.931 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 5 to smithi11731613-1 2024-09-16T06:49:03.932 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 6 to smithi11731613-1 2024-09-16T06:49:03.934 INFO:tasks.rados.rados.0.smithi117.stdout:264: finishing write tid 7 to smithi11731613-1 2024-09-16T06:49:03.934 INFO:tasks.rados.rados.0.smithi117.stdout:264: oid 1 updating version 0 to 69 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:264: oid 1 version 69 is already newer than 68 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:264: oid 1 updating version 69 to 71 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:264: oid 1 updating version 71 to 72 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:264: oid 1 updating version 72 to 73 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:264: oid 1 version 73 is already newer than 70 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 1 v 73 (ObjNum 119 snap 27 seq_num 119) dirty exists 2024-09-16T06:49:03.935 INFO:tasks.rados.rados.0.smithi117.stdout:264: left oid 1 (ObjNum 119 snap 27 seq_num 119) 2024-09-16T06:49:03.939 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:03.940 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:03.984 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:03.984 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:03.984 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:03.998 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:49:04.025 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:04.024+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:04.127 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:04.123+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:04.166 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:266 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.210487129999997, 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014774862999999999, 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:04.167 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 266 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:04.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953825+0000", 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953846+0000", 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0171999999999999e-05 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953889+0000", 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3284000000000001e-05 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953913+0000", 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4071e-05 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954055+0000", 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014240699999999999 2024-09-16T06:49:04.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954076+0000", 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0678000000000001e-05 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961148+0000", 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070716959999999997 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961402+0000", 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025470100000000003 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.968604+0000", 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072013340000000002 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.172 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:267 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.210370918000002, 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015485804000000001, 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 267 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953944+0000", 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953973+0000", 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9581e-05 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953995+0000", 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1685e-05 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954011+0000", 2024-09-16T06:49:04.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5862000000000002e-05 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954223+0000", 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021187199999999999 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954233+0000", 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0789e-05 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962540+0000", 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0083068819999999998 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962710+0000", 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017007799999999999 2024-09-16T06:49:04.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.969431+0000", 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0067204250000000004 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:268 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.210290671000003, 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.30266284300000001, 2024-09-16T06:49:04.178 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 268 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954024+0000", 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954032+0000", 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4290000000000004e-06 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954050+0000", 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8332e-05 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954063+0000", 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2955999999999999e-05 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954379+0000", 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031616000000000001 2024-09-16T06:49:04.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954389+0000", 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3959999999999995e-06 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963428+0000", 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0090396959999999998 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963723+0000", 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00029431800000000001 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:29.256688+0000", 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.29296564200000003 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:49:04.183 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.329028481999998, 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014247134, 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:04.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835286+0000", 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835349+0000", 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3258999999999999e-05 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835459+0000", 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110097 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835480+0000", 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1424999999999999e-05 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835600+0000", 2024-09-16T06:49:04.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011922 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835623+0000", 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3144999999999999e-05 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849454+0000", 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013830779 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849535+0000", 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1021999999999998e-05 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 61/32 ECSubWrite(tid=83, reqid=osd.14.0:83, at_version=61'72, trim_to=0'0, roll_forward_to=59'68))", 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 15.330743305, 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012777472999999999, 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.188 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833571+0000", 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833631+0000", 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9917999999999998e-05 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833761+0000", 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013055799999999999 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833784+0000", 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2714999999999999e-05 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835926+0000", 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021422720000000002 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835943+0000", 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6889e-05 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846281+0000", 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010337641999999999 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846350+0000", 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9315999999999998e-05 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.192 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 61/32 ECSubWrite(tid=85, reqid=osd.10.0:85, at_version=61'70, trim_to=0'0, roll_forward_to=59'66))", 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 15.325861572000001, 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013138069, 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838452+0000", 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838506+0000", 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3594999999999998e-05 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839270+0000", 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00076359400000000003 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839293+0000", 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3006e-05 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839420+0000", 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127355 2024-09-16T06:49:04.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839446+0000", 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851525+0000", 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012079196 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851592+0000", 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7156999999999994e-05 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=101, reqid=osd.10.0:101, at_version=72'86, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3268483780000002, 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017459348, 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.197 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837467+0000", 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837538+0000", 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0541000000000001e-05 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837664+0000", 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012650899999999999 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837687+0000", 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2759999999999999e-05 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839677+0000", 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019901300000000001 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839707+0000", 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0049e-05 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854862+0000", 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015155366 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854927+0000", 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4629999999999996e-05 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=157, reqid=osd.3.0:157, at_version=72'123, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3265354250000003, 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027234286, 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837780+0000", 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837839+0000", 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8801000000000002e-05 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837959+0000", 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000120436 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837982+0000", 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3156e-05 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838118+0000", 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000135673 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838143+0000", 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4899e-05 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.864948+0000", 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026805539999999999 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865015+0000", 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6501000000000006e-05 2024-09-16T06:49:04.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=102, reqid=osd.10.0:102, at_version=72'88, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3237244409999995, 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018351104, 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:04.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840590+0000", 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840640+0000", 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0631999999999999e-05 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840728+0000", 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7933e-05 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840750+0000", 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2022999999999999e-05 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840864+0000", 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113423 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840889+0000", 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5092999999999998e-05 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858883+0000", 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017994231999999999 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858943+0000", 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9587999999999999e-05 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=158, reqid=osd.3.0:158, at_version=72'125, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3220833040000004, 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022836532999999999, 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.210 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842231+0000", 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842295+0000", 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4238000000000004e-05 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842418+0000", 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012244299999999999 2024-09-16T06:49:04.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842441+0000", 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3606000000000001e-05 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850726+0000", 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082850219999999995 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850750+0000", 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3822e-05 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865034+0000", 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014283563000000001 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865069+0000", 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5712e-05 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=65, reqid=osd.12.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3187602790000001, 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017333787, 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845555+0000", 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845609+0000", 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4284999999999999e-05 2024-09-16T06:49:04.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845766+0000", 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015658100000000001 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845789+0000", 2024-09-16T06:49:04.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2866000000000001e-05 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845942+0000", 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015326200000000001 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845969+0000", 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7296000000000001e-05 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862827+0000", 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016857351999999999 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862890+0000", 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2997999999999996e-05 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=66, reqid=osd.12.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3170687929999998, 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017965090999999999, 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847246+0000", 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847298+0000", 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2677999999999999e-05 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847412+0000", 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113403 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847437+0000", 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4973e-05 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857022+0000", 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0095850889999999998 2024-09-16T06:49:04.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857045+0000", 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3796000000000001e-05 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865168+0000", 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0081221260000000003 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865212+0000", 2024-09-16T06:49:04.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4691999999999998e-05 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=108, reqid=osd.14.0:108, at_version=72'100, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3139697809999999, 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015047932999999999, 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850339+0000", 2024-09-16T06:49:04.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999928 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850413+0000", 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3508000000000002e-05 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850503+0000", 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0303999999999995e-05 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850526+0000", 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2595999999999998e-05 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850694+0000", 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168367 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850718+0000", 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.44e-05 2024-09-16T06:49:04.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865344+0000", 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014625658999999999 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865394+0000", 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0074999999999997e-05 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=109, reqid=osd.14.0:109, at_version=72'102, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.3103222130000001, 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013551083, 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:04.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853992+0000", 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854050+0000", 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8495000000000002e-05 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854252+0000", 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020159999999999999 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854296+0000", 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4438999999999999e-05 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862592+0000", 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082959690000000003 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862662+0000", 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9210999999999999e-05 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867494+0000", 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048324029999999999 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867545+0000", 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0838999999999999e-05 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:558 3.5s0 3:af79444c:::smithi11731613-29:1d [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.954915+0000", 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.2094008030000003, 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012069846, 2024-09-16T06:49:04.233 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 558 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954915+0000", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954915+0000", 2024-09-16T06:49:04.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954915+0000", 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954920+0000", 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4e-06 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954929+0000", 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3370000000000006e-06 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954936+0000", 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7580000000000003e-06 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.963228+0000", 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082919020000000003 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.963397+0000", 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016865200000000001 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.966985+0000", 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035884519999999998 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.237 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:559 3.5s0 3:af79444c:::smithi11731613-29:1d [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.2093360549999996, 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014695526, 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 559 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954979+0000", 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954984+0000", 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5310000000000003e-06 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954994+0000", 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7249999999999992e-06 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.955000+0000", 2024-09-16T06:49:04.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7569999999999998e-06 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967068+0000", 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012068096 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967239+0000", 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017012400000000001 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.969675+0000", 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024368660000000002 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=191, reqid=osd.6.0:191, at_version=73'31, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848719+0000", 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.3155967840000002, 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01207479, 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848719+0000", 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848719+0000", 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848718+0000", 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848779+0000", 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0133999999999997e-05 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848977+0000", 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00019813099999999999 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849037+0000", 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9808000000000001e-05 2024-09-16T06:49:04.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849244+0000", 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020772999999999999 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849271+0000", 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6652000000000002e-05 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.860732+0000", 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011461176999999999 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.860794+0000", 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.1897999999999996e-05 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=192, reqid=osd.6.0:192, at_version=73'33, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.31552997, 2024-09-16T06:49:04.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017679872999999999, 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848785+0000", 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848820+0000", 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4931999999999997e-05 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849087+0000", 2024-09-16T06:49:04.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026683499999999999 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849103+0000", 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6813e-05 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855706+0000", 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066022570000000003 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855730+0000", 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4661999999999999e-05 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866426+0000", 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010695893999999999 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866466+0000", 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9632999999999998e-05 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.3134040680000001, 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015506908999999999, 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850911+0000", 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850952+0000", 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0722000000000001e-05 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851117+0000", 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000164599 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851132+0000", 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5255e-05 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854555+0000", 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034234320000000001 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854579+0000", 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866327+0000", 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011748364000000001 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866419+0000", 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1624000000000002e-05 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 73/32 ECSubWrite(tid=107, reqid=osd.10.0:107, at_version=73'90, trim_to=0'0, roll_forward_to=72'88))", 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.3101515969999999, 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012199229000000001, 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:04.255 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:04.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854217+0000", 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3168000000000003e-05 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854327+0000", 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109833 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854349+0000", 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2296000000000001e-05 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855253+0000", 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090369500000000004 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855283+0000", 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9884999999999999e-05 2024-09-16T06:49:04.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866242+0000", 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01095932 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866364+0000", 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000121542 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.259 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:04.260 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:04.260 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:04.261 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-16T06:49:04.180+0000 7fce2dab6640 -1 mon.a@0(leader).osd e76 definitely_dead 0 2024-09-16T06:49:04.262 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:04.255+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:04.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:49:04.275 INFO:tasks.rados.rados.0.smithi117.stdout:262: finishing write tid 5 to smithi11731613-34 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: finishing write tid 6 to smithi11731613-34 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: oid 34 updating version 0 to 26 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: oid 34 version 26 is already newer than 24 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: oid 34 updating version 26 to 27 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: oid 34 updating version 27 to 28 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: oid 34 version 28 is already newer than 25 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 34 v 28 (ObjNum 117 snap 27 seq_num 117) dirty exists 2024-09-16T06:49:04.278 INFO:tasks.rados.rados.0.smithi117.stdout:262: left oid 34 (ObjNum 117 snap 27 seq_num 117) 2024-09-16T06:49:04.362 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:04.359+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:04.384 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:04.384 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:04.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:49:04.463 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:04.459+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:04.521 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:04.521 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:04.521 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:04.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:49:04.565 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:04.563+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:04.653 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.653 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:135 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'93, mlcod=25'89)", 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.179911975000003, 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073860260000000004, 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476337+0000", 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476374+0000", 2024-09-16T06:49:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7509000000000003e-05 2024-09-16T06:49:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476453+0000", 2024-09-16T06:49:04.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8273000000000001e-05 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476468+0000", 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476574+0000", 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000106879 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476624+0000", 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9910000000000002e-05 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483640+0000", 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00701565 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483724+0000", 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4535000000000003e-05 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.658 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:141 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'99, mlcod=25'91)", 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.170408168000002, 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.28011762800000001, 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485841+0000", 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485852+0000", 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1313e-05 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764267+0000", 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.278414628 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764288+0000", 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.105e-05 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764448+0000", 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016007599999999999 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764487+0000", 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9125999999999998e-05 2024-09-16T06:49:04.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765872+0000", 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001384763 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765960+0000", 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7979000000000006e-05 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:160 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'114, mlcod=26'107)", 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.726637238999999, 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061551990000000001, 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929613+0000", 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929617+0000", 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5430000000000004e-06 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929973+0000", 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035563700000000001 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929979+0000", 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7599999999999999e-06 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.931187+0000", 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001208081 2024-09-16T06:49:04.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.931239+0000", 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2522999999999998e-05 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935719+0000", 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044799219999999999 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935768+0000", 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9113000000000002e-05 2024-09-16T06:49:04.667 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:161 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'115, mlcod=26'107)", 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.726630395999997, 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062540549999999997, 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929715+0000", 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5147000000000002e-05 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929991+0000", 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027581399999999999 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929997+0000", 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1399999999999997e-06 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932071+0000", 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020736499999999998 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932100+0000", 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9090000000000001e-05 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935772+0000", 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0036722019999999998 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935874+0000", 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102296 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:162 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'116, mlcod=26'107)", 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:49:04.672 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.726531323000003, 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062203090000000003, 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929719+0000", 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929791+0000", 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1872999999999994e-05 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930051+0000", 2024-09-16T06:49:04.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026016400000000001 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930059+0000", 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4100000000000008e-06 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933018+0000", 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029584709999999998 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933048+0000", 2024-09-16T06:49:04.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.029e-05 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935882+0000", 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028339509999999999 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935939+0000", 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7636999999999997e-05 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:163 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'117, mlcod=26'107)", 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.726453304000003, 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062254820000000001, 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929801+0000", 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2030000000000002e-06 2024-09-16T06:49:04.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930091+0000", 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028969100000000002 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930097+0000", 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1870000000000002e-06 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933566+0000", 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003469663 2024-09-16T06:49:04.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933600+0000", 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3293000000000001e-05 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935942+0000", 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023424190000000001 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936023+0000", 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0438999999999993e-05 2024-09-16T06:49:04.680 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:164 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'118, mlcod=26'107)", 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.726446860999999, 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062774559999999998, 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929803+0000", 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929846+0000", 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3038999999999999e-05 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930109+0000", 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026273799999999999 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930115+0000", 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0900000000000001e-06 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934327+0000", 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004212248 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934363+0000", 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5576e-05 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936028+0000", 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016651770000000001 2024-09-16T06:49:04.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936081+0000", 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2874000000000002e-05 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:165 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'119, mlcod=26'107)", 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.726401117999998, 2024-09-16T06:49:04.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0069224810000000003, 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929854+0000", 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7600000000000002e-06 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930127+0000", 2024-09-16T06:49:04.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000273374 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930135+0000", 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.157e-06 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935309+0000", 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051733370000000001 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935340+0000", 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1423000000000001e-05 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936696+0000", 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013555730000000001 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936772+0000", 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6260000000000005e-05 2024-09-16T06:49:04.689 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=32, reqid=client.4511.0:237, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.694125133, 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.008178467, 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962124+0000", 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962148+0000", 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3572999999999999e-05 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962287+0000", 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013903200000000001 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962295+0000", 2024-09-16T06:49:04.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1999999999999994e-06 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962430+0000", 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135129 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962474+0000", 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3503e-05 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970216+0000", 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077425230000000003 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7330000000000006e-05 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.694 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=34, reqid=client.4511.0:228, at_version=40'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.693376403000002, 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075127500000000003, 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962872+0000", 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962920+0000", 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7939999999999998e-05 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963011+0000", 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0655999999999994e-05 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963034+0000", 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3782999999999999e-05 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964104+0000", 2024-09-16T06:49:04.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010698509999999999 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964127+0000", 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2563000000000001e-05 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061773169999999999 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970387+0000", 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2680000000000001e-05 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=35, reqid=client.4511.0:238, at_version=40'13, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:04.699 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.692944730000001, 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00704997, 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963304+0000", 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963342+0000", 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7753000000000003e-05 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963421+0000", 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9549999999999996e-05 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963436+0000", 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4885999999999999e-05 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964389+0000", 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095243900000000002 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964403+0000", 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.381e-05 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970320+0000", 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059176259999999996 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970356+0000", 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5525999999999998e-05 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.703 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=36, reqid=client.4511.0:229, at_version=40'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.692436440999998, 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006602887, 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963812+0000", 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963860+0000", 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7182999999999999e-05 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963954+0000", 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3912999999999993e-05 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963975+0000", 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1506e-05 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964846+0000", 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087055200000000002 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964868+0000", 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.243e-05 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970396+0000", 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005528397 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970417+0000", 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0418999999999999e-05 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=37, reqid=client.4511.0:239, at_version=40'14, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.692242489000002, 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070022000000000001, 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964007+0000", 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964083+0000", 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6262999999999998e-05 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964162+0000", 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8568999999999999e-05 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964181+0000", 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9283000000000002e-05 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965031+0000", 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00084985599999999996 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965056+0000", 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4868999999999999e-05 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970969+0000", 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059132270000000001 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.971010+0000", 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1285999999999997e-05 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.712 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=77, reqid=osd.1.0:77, at_version=52'38, trim_to=0'0, roll_forward_to=50'36))", 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 24.794097971999999, 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061672749999999998, 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862152+0000", 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862151+0000", 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862203+0000", 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2352000000000003e-05 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862291+0000", 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8163000000000003e-05 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862313+0000", 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2255999999999998e-05 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862436+0000", 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000122753 2024-09-16T06:49:04.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.862462+0000", 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5559999999999999e-05 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868253+0000", 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057913009999999996 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.868320+0000", 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6855999999999998e-05 2024-09-16T06:49:04.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=80, reqid=osd.1.0:78, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 24.776119514000001, 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064748710000000001, 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:49:04.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880130+0000", 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880152+0000", 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2322000000000001e-05 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880228+0000", 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6367999999999998e-05 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880245+0000", 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6186999999999999e-05 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880330+0000", 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.5568000000000005e-05 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880347+0000", 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6528000000000002e-05 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886553+0000", 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006206423 2024-09-16T06:49:04.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886606+0000", 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2704999999999998e-05 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/37 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'37, trim_to=0'0, roll_forward_to=50'35))", 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 24.774573072999999, 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0081001449999999996, 2024-09-16T06:49:04.721 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881676+0000", 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881715+0000", 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8512e-05 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881760+0000", 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4669999999999998e-05 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881776+0000", 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6371999999999998e-05 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881968+0000", 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019165999999999999 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881998+0000", 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0332999999999999e-05 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889735+0000", 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077367310000000002 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889778+0000", 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2766999999999999e-05 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.725 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=81, reqid=osd.1.0:81, at_version=52'42, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 24.773591689, 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076324610000000001, 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882657+0000", 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882697+0000", 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9900000000000001e-05 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882763+0000", 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5949000000000002e-05 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882779+0000", 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6118000000000001e-05 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883074+0000", 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000294393 2024-09-16T06:49:04.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883098+0000", 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4049999999999998e-05 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890245+0000", 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071470479999999996 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890291+0000", 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6343e-05 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/32 ECSubWrite(tid=177, reqid=osd.6.0:177, at_version=61'18, trim_to=0'0, roll_forward_to=57'16))", 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.819837+0000", 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 15.836413546999999, 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078136230000000004, 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.730 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819837+0000", 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819837+0000", 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819836+0000", 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819882+0000", 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6022000000000001e-05 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819984+0000", 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010180599999999999 2024-09-16T06:49:04.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820009+0000", 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4746e-05 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820373+0000", 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036459699999999998 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820399+0000", 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6060000000000001e-05 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.827615+0000", 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072153490000000002 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.827651+0000", 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5709e-05 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 61/37 ECSubWrite(tid=90, reqid=osd.1.0:90, at_version=61'54, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.820306+0000", 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 15.835944316999999, 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061244139999999999, 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820306+0000", 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820306+0000", 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820305+0000", 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820352+0000", 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7162e-05 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820431+0000", 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9289999999999995e-05 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820452+0000", 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0432999999999999e-05 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820575+0000", 2024-09-16T06:49:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012363200000000001 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820602+0000", 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6176000000000001e-05 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.826369+0000", 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057674809999999996 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.826431+0000", 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1539999999999997e-05 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.823369+0000", 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 15.832881181999999, 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086286320000000007, 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:04.739 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823369+0000", 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823369+0000", 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:04.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823368+0000", 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823392+0000", 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4474999999999999e-05 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823437+0000", 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4465999999999997e-05 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823447+0000", 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0073e-05 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823566+0000", 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118724 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823595+0000", 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9471999999999998e-05 2024-09-16T06:49:04.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831950+0000", 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0083552839999999993 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831998+0000", 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7636000000000003e-05 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:04.743 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:04.744 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:04.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:49:04.746 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:04.743+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:04.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:49:04.846 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:04.846+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:04.869 INFO:teuthology.orchestra.run.smithi100.stderr:marked in osd.13. 2024-09-16T06:49:04.871 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:04.871 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:04.871 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:04.876 INFO:tasks.rados.rados.0.smithi117.stdout:262: done (2 left) 2024-09-16T06:49:04.876 INFO:tasks.rados.rados.0.smithi117.stdout:264: done (1 left) 2024-09-16T06:49:04.876 INFO:tasks.rados.rados.0.smithi117.stdout:265: done (0 left) 2024-09-16T06:49:04.876 INFO:tasks.rados.rados.0.smithi117.stdout:266: delete oid 28 current snap is 28 2024-09-16T06:49:04.895 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:49:04.897 INFO:tasks.thrashosds.thrasher:Added osd 13 2024-09-16T06:49:04.983 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:04.984+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:05.048 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:05.047+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:05.054 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:05.054 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:05.055 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:05.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:49:05.149 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:05.147+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:05.233 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:05.233 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:05.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=12, reqid=client.4511.0:68, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.273381696999998, 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.041915173, 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961089+0000", 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.235 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961108+0000", 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8260000000000001e-05 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001032+0000", 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.039924492999999998 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001043+0000", 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1063000000000001e-05 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001095+0000", 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1703999999999998e-05 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001102+0000", 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8360000000000003e-06 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002963+0000", 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018614529999999999 2024-09-16T06:49:05.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.003005+0000", 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2182999999999999e-05 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=14, reqid=client.4511.0:98, at_version=35'10, trim_to=0'0, roll_forward_to=35'7))", 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.232307945000002, 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082808355, 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.238 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002163+0000", 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002178+0000", 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4431e-05 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084421+0000", 2024-09-16T06:49:05.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082243256000000001 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084430+0000", 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8880000000000003e-06 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084484+0000", 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4193000000000001e-05 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084496+0000", 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1754e-05 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084946+0000", 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000450242 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084972+0000", 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6281000000000001e-05 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.242 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=16, reqid=client.4511.0:158, at_version=35'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.150087816000003, 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41790576000000001, 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084383+0000", 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084401+0000", 2024-09-16T06:49:05.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7584999999999999e-05 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500734+0000", 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41633377199999999 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500752+0000", 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7374e-05 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500862+0000", 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010999500000000001 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500877+0000", 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5437000000000001e-05 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502236+0000", 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001358389 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502290+0000", 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4076000000000002e-05 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:252 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 37.170955065000001, 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.38343603199999998, 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.247 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 252 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063515+0000", 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063523+0000", 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7430000000000002e-06 2024-09-16T06:49:05.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063545+0000", 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1310000000000001e-05 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063562+0000", 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7629999999999999e-05 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068202+0000", 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046400269999999997 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068467+0000", 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000264691 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070315+0000", 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018477190000000001 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070443+0000", 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012795699999999999 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.446953+0000", 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.376510228 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:482 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~442368] snapc 1a=[1a,19,18,15,11,c,8,5] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:49:05.252 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 16.384144829, 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019730733, 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 482 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850326+0000", 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851134+0000", 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00080810400000000003 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851148+0000", 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3689000000000001e-05 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851153+0000", 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8329999999999999e-06 2024-09-16T06:49:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851191+0000", 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8087000000000003e-05 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851262+0000", 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0964999999999996e-05 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851984+0000", 2024-09-16T06:49:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00072110499999999999 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.859456+0000", 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0074722319999999997 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.869982+0000", 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010525806 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870058+0000", 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6180000000000006e-05 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:511 3.7s0 3:e8f80969:::smithi11731613-49:head [append 0~442368] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:52.882250+0000", 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.352221953000001, 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.017244913000000001, 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.258 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 511 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882250+0000", 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882250+0000", 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882248+0000", 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882792+0000", 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00054419799999999997 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882812+0000", 2024-09-16T06:49:05.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9802000000000001e-05 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882820+0000", 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2649999999999996e-06 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882871+0000", 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0850000000000003e-05 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.883044+0000", 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000173031 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.883874+0000", 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000830089 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.885467+0000", 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001593115 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.899385+0000", 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013918216000000001 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.899495+0000", 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000109623 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:512 3.7s0 3:e8f80969:::smithi11731613-49:head [append 0~442368] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:49:05.263 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:52.887502+0000", 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 12.346970057, 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015186880999999999, 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 512 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887502+0000", 2024-09-16T06:49:05.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887502+0000", 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887501+0000", 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887896+0000", 2024-09-16T06:49:05.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039566100000000001 2024-09-16T06:49:05.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887910+0000", 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4154999999999999e-05 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887919+0000", 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1240000000000005e-06 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887958+0000", 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9740000000000002e-05 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.888046+0000", 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8164000000000005e-05 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.888736+0000", 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068976699999999998 2024-09-16T06:49:05.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.890458+0000", 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001721658 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.902594+0000", 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.012136378999999999 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.902689+0000", 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4430000000000002e-05 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:549 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.899393+0000", 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.3350790220000004, 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.055164967000000002, 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.271 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 549 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899393+0000", 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899393+0000", 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899392+0000", 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899397+0000", 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5399999999999997e-06 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899403+0000", 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7870000000000004e-06 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899409+0000", 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1970000000000004e-06 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.900891+0000", 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001482627 2024-09-16T06:49:05.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.900979+0000", 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8354000000000001e-05 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.901408+0000", 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042863200000000003 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.902446+0000", 2024-09-16T06:49:05.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001038244 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954441+0000", 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.051994264999999998 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954558+0000", 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117141 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:550 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.899430+0000", 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.3350419960000002, 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.057651908000000002, 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.277 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 550 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899430+0000", 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899430+0000", 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899429+0000", 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899453+0000", 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3285000000000002e-05 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899462+0000", 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0599999999999997e-06 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899469+0000", 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.934e-06 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.902346+0000", 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028773259999999999 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.902438+0000", 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1891999999999994e-05 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954593+0000", 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.052154813000000001 2024-09-16T06:49:05.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954665+0000", 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2537e-05 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957082+0000", 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024165699999999998 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.282 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:551 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~540672] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.948074+0000", 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.2863976319999999, 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.017190535, 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 551 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948074+0000", 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948074+0000", 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948071+0000", 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948505+0000", 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00043384000000000001 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948537+0000", 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1826000000000003e-05 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948552+0000", 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:49:05.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948653+0000", 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010173199999999999 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948828+0000", 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000174289 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950509+0000", 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016809049999999999 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957393+0000", 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0068844889999999997 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.965043+0000", 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076495920000000002 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.965265+0000", 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022198199999999999 2024-09-16T06:49:05.287 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:555 3.7s0 3:e6e9cc11:::smithi11731613-37:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.958998+0000", 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.2754738830000001, 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.047641277000000003, 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.288 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 555 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.958998+0000", 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.958998+0000", 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.958997+0000", 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959001+0000", 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0819999999999999e-06 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959005+0000", 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4660000000000001e-06 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959009+0000", 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.985e-06 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.967578+0000", 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0085692860000000006 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.967871+0000", 2024-09-16T06:49:05.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00029280099999999998 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.969068+0000", 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00119688 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.970919+0000", 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001850824 2024-09-16T06:49:05.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006489+0000", 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.035569890999999999 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006639+0000", 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000150538 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.293 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:556 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.959035+0000", 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.2754370589999997, 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.052626567999999999, 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 556 2024-09-16T06:49:05.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959035+0000", 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959035+0000", 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959034+0000", 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959036+0000", 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9719999999999999e-06 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959041+0000", 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2479999999999998e-06 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959044+0000", 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2310000000000001e-06 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.969877+0000", 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010833746999999999 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.970197+0000", 2024-09-16T06:49:05.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031932100000000001 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006695+0000", 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.036498435000000003 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006765+0000", 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0025999999999997e-05 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.011661+0000", 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004896063 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:569 3.7s0 3:e8f80969:::smithi11731613-49:head [append 0~573440] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.000711+0000", 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.23376062, 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015629909000000001, 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.299 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 569 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.000711+0000", 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.000711+0000", 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.000702+0000", 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999905 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001202+0000", 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050003299999999999 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001230+0000", 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7991999999999999e-05 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001244+0000", 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3890000000000001e-05 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001306+0000", 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2623e-05 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001432+0000", 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000125344 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.002725+0000", 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012930839999999999 2024-09-16T06:49:05.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.004929+0000", 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002203881 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016214+0000", 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011285394000000001 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016341+0000", 2024-09-16T06:49:05.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127055 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:572 3.7s0 3:e8f80969:::smithi11731613-49:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.2250153959999999, 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.30836179800000002, 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 572 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009463+0000", 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6830000000000008e-06 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009473+0000", 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0052e-05 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009482+0000", 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9570000000000008e-06 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.014506+0000", 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050242560000000004 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.014686+0000", 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017992299999999999 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.015477+0000", 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00079010899999999999 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016467+0000", 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099052600000000004 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.317653+0000", 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.30118623999999999 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.317818+0000", 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016493200000000001 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.310 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:573 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.2249836509999996, 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.31066434700000001, 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 573 2024-09-16T06:49:05.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009491+0000", 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1499999999999999e-06 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009501+0000", 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.006e-05 2024-09-16T06:49:05.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009508+0000", 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5920000000000003e-06 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016668+0000", 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071597359999999999 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016752+0000", 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4153999999999994e-05 2024-09-16T06:49:05.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.317902+0000", 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.30114975199999999 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.318063+0000", 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016121199999999999 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.320153+0000", 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020892990000000002 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:598 3.7s0 3:e60b6e1b:::smithi11731613-14:head [stat,delete] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.304558+0000", 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.9299142150000002, 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.02248083, 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.316 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 598 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304558+0000", 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304558+0000", 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304555+0000", 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304569+0000", 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4270000000000001e-05 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304602+0000", 2024-09-16T06:49:05.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2725999999999998e-05 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304619+0000", 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6759999999999999e-05 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304707+0000", 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8028999999999994e-05 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304878+0000", 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017184500000000001 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.305786+0000", 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00090726900000000004 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.307268+0000", 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0014825280000000001 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.326936+0000", 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019667604000000002 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.327038+0000", 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102599 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:600 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~458752] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.581230+0000", 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.6532418560000002, 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015383671, 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 600 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581230+0000", 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581230+0000", 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581227+0000", 2024-09-16T06:49:05.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581694+0000", 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046673599999999998 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581728+0000", 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4564000000000003e-05 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581745+0000", 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6322999999999999e-05 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.582372+0000", 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000627051 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.582680+0000", 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030799100000000003 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.584495+0000", 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001815662 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587062+0000", 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002566774 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.596421+0000", 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0093590089999999997 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.596614+0000", 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019236500000000001 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:602 3.7s0 3:e6e9cc11:::smithi11731613-37:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.587820+0000", 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.6466518509999997, 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014926857, 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 602 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.328 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587820+0000", 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587820+0000", 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587819+0000", 2024-09-16T06:49:05.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587826+0000", 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5870000000000004e-06 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587836+0000", 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0295999999999999e-05 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587845+0000", 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2099999999999993e-06 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.593779+0000", 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059341139999999999 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.594040+0000", 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026153100000000001 2024-09-16T06:49:05.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.595212+0000", 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001171657 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.601503+0000", 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062912419999999998 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.601701+0000", 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019831499999999999 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.602747+0000", 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010453719999999999 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:603 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.587875+0000", 2024-09-16T06:49:05.333 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.6465966019999998, 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019344, 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 603 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587875+0000", 2024-09-16T06:49:05.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587875+0000", 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587874+0000", 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587879+0000", 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5129999999999998e-06 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587891+0000", 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1684e-05 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587902+0000", 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1562999999999999e-05 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.597930+0000", 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010027968999999999 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.598199+0000", 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000268857 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.602859+0000", 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046601280000000004 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.603117+0000", 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025791099999999999 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.607219+0000", 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041022020000000001 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 73/37 ECSubWriteReply(tid=95, last_complete=73'88, committed=1, applied=1))", 2024-09-16T06:49:05.338 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.3639516560000002, 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.083448052999999994, 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870525+0000", 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3789999999999997e-06 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.952902+0000", 2024-09-16T06:49:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082376648999999996 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.952915+0000", 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3373000000000001e-05 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.952993+0000", 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8119000000000002e-05 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.953968+0000", 2024-09-16T06:49:05.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00097517899999999995 2024-09-16T06:49:05.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.342 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.342 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.342 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:05.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:49:05.344 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:05.348+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:05.444 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:05.444+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:05.513 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:05.513 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:05.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:49:05.545 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:05.543+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:05.646 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:05.643+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:05.691 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:05.691 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:05.691 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:05.692 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:05.692 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:05.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:49:05.747 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:05.746+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:49:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:52 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.895674800999998, 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011921681999999999, 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.831 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 52 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:49:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936107+0000", 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936869+0000", 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076224400000000003 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936883+0000", 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3704e-05 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936893+0000", 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0567000000000001e-05 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937221+0000", 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032796600000000001 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937632+0000", 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00041103599999999999 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939883+0000", 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022507009999999999 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943370+0000", 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034869190000000002 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.947880+0000", 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045105839999999998 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.948030+0000", 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015010800000000001 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.836 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:53 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.894873527000001, 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.679887028, 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.837 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 53 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936909+0000", 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936915+0000", 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8130000000000001e-06 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936921+0000", 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9939999999999997e-06 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936925+0000", 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9920000000000001e-06 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941426+0000", 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045008390000000004 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941657+0000", 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023140500000000001 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.942737+0000", 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001079637 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943730+0000", 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099337199999999996 2024-09-16T06:49:05.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616662+0000", 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67293178399999998 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616797+0000", 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135186 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:54 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.894855292999999, 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2642334100000001, 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 54 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936930+0000", 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.725e-06 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936933+0000", 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.698e-06 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936967+0000", 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3618000000000002e-05 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943520+0000", 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065532869999999997 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943693+0000", 2024-09-16T06:49:05.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017226899999999999 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617527+0000", 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67383411699999995 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617765+0000", 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023803499999999999 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201162+0000", 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58339697800000001 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.es0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.212805437999997, 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58220860799999996, 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.848 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618977+0000", 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618996+0000", 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9266000000000002e-05 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200456+0000", 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58146017500000002 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200480+0000", 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3186000000000001e-05 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200604+0000", 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012390599999999999 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201187+0000", 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00058315500000000002 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 40/37 ECSubWrite(tid=20, reqid=client.4511.0:219, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 37.850911955000001, 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005811274, 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980872+0000", 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.980871+0000", 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981331+0000", 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046021300000000001 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981469+0000", 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138088 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981496+0000", 2024-09-16T06:49:05.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7353000000000001e-05 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981653+0000", 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000156816 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.981681+0000", 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.73e-05 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986615+0000", 2024-09-16T06:49:05.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049340180000000001 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.986683+0000", 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8258999999999998e-05 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:433 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~737280] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:49:05.856 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.879408721000001, 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013805337000000001, 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 433 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952372+0000", 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953751+0000", 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013786060000000001 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953786+0000", 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5605999999999998e-05 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953802+0000", 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5478999999999999e-05 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953976+0000", 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000173979 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.954152+0000", 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017637499999999999 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.957075+0000", 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029234159999999999 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.960418+0000", 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003342579 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965998+0000", 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055795719999999997 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966180+0000", 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000182578 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:434 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~573440] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.870028742999999, 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011138353, 2024-09-16T06:49:05.862 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 434 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961752+0000", 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962728+0000", 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00097593799999999998 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962755+0000", 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6597e-05 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962769+0000", 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3583e-05 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962853+0000", 2024-09-16T06:49:05.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4399000000000003e-05 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.963021+0000", 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000167625 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964326+0000", 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013054519999999999 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972047+0000", 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007721042 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972718+0000", 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00067065799999999997 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972893+0000", 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000175472 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.867 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:435 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~196608] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.866807497, 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086272310000000008, 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.868 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 435 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964973+0000", 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965356+0000", 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038219399999999998 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965388+0000", 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2276000000000001e-05 2024-09-16T06:49:05.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965403+0000", 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966329+0000", 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00092638200000000001 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966612+0000", 2024-09-16T06:49:05.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000282525 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.968300+0000", 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016878450000000001 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973229+0000", 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049290990000000002 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973498+0000", 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026954399999999998 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973603+0000", 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105053 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.873 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:436 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (57) in=7b] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.866368031, 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.01058251, 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 436 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965414+0000", 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965422+0000", 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.43e-06 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965435+0000", 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3332999999999999e-05 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965445+0000", 2024-09-16T06:49:05.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4800000000000007e-06 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.969503+0000", 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0040581259999999996 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.969750+0000", 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024736799999999998 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.970629+0000", 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087915900000000002 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975095+0000", 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044660719999999998 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975865+0000", 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076922299999999998 2024-09-16T06:49:05.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975998+0000", 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000133442 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:437 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:05.879 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.866329961000002, 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014457724, 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 437 2024-09-16T06:49:05.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965453+0000", 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965457+0000", 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0500000000000002e-06 2024-09-16T06:49:05.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965471+0000", 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4403000000000001e-05 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965479+0000", 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7859999999999997e-06 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.974299+0000", 2024-09-16T06:49:05.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0088199260000000005 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.974438+0000", 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000139519 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.976547+0000", 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021088449999999998 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.976785+0000", 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023817099999999999 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.979911+0000", 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031258570000000001 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 55/37 ECSubWriteReply(tid=6, last_complete=55'5, committed=1, applied=1))", 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:49:05.886 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.865037758, 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061836670000000003, 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:49:05.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966745+0000", 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966753+0000", 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9759999999999995e-06 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966796+0000", 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3655999999999997e-05 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966805+0000", 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1200000000000008e-06 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972502+0000", 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056967390000000001 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972929+0000", 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000427253 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 61/37 ECSubWrite(tid=39, reqid=osd.4.0:39, at_version=61'34, trim_to=0'0, roll_forward_to=58'32))", 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 17.006541932000001, 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059543850000000004, 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.890 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825241+0000", 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825303+0000", 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1846000000000004e-05 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825467+0000", 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000164575 2024-09-16T06:49:05.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825495+0000", 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7705999999999999e-05 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.828570+0000", 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030753009999999999 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.828603+0000", 2024-09-16T06:49:05.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2546999999999999e-05 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831167+0000", 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002564588 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831196+0000", 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8532000000000001e-05 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.894 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 61/54 ECSubWrite(tid=121, reqid=osd.3.0:121, at_version=61'51, trim_to=0'0, roll_forward_to=57'47))", 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 16.983102747, 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063343469999999997, 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.895 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848679+0000", 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848734+0000", 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5569999999999998e-05 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848826+0000", 2024-09-16T06:49:05.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1281999999999998e-05 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848851+0000", 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4837e-05 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848916+0000", 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5440999999999999e-05 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848926+0000", 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7359999999999996e-06 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854992+0000", 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060664559999999996 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.855015+0000", 2024-09-16T06:49:05.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2955e-05 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/37 ECSubWriteReply(tid=16, last_complete=61'13, committed=1, applied=1))", 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 16.976927044, 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.246763658, 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854856+0000", 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:05.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854862+0000", 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7479999999999999e-06 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101354+0000", 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.24649253400000001 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101371+0000", 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7116e-05 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101507+0000", 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013554600000000001 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101620+0000", 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011343199999999999 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.903 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:518 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1802240,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:54.894072+0000", 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 10.937711688, 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.024146883000000001, 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 518 2024-09-16T06:49:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894072+0000", 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894072+0000", 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894068+0000", 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894088+0000", 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0106999999999999e-05 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894125+0000", 2024-09-16T06:49:05.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6508999999999999e-05 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894143+0000", 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8476999999999999e-05 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894256+0000", 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112626 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op must wait for map", 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894270+0000", 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4600000000000001e-05 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.897757+0000", 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00348702 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.897904+0000", 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146539 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.918219+0000", 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.020314855 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:519 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1802240,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:54.894164+0000", 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 10.937619998000001, 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.025821218, 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.909 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 519 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894164+0000", 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894164+0000", 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894162+0000", 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894171+0000", 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0329999999999997e-06 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894191+0000", 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0046000000000001e-05 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894201+0000", 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6900000000000004e-06 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894425+0000", 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022422800000000001 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894444+0000", 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8377e-05 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.898809+0000", 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043656099999999998 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.898985+0000", 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017558100000000001 2024-09-16T06:49:05.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.919985+0000", 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.020999903 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:520 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1802240,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:54.894243+0000", 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 10.937540466, 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.027308196999999999, 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:05.914 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 520 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894243+0000", 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894243+0000", 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894241+0000", 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894252+0000", 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0319999999999999e-05 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894272+0000", 2024-09-16T06:49:05.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0526e-05 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894288+0000", 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5644000000000001e-05 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894528+0000", 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000239764 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:05.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894544+0000", 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6577000000000001e-05 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.900957+0000", 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064123060000000004 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.901095+0000", 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013892900000000001 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.921551+0000", 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.020455804000000001 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 72/37 ECSubWrite(tid=72, reqid=client.4511.0:551, at_version=72'68, trim_to=0'0, roll_forward_to=72'65))", 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.950678+0000", 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.8811058379999999, 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070099439999999997, 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950678+0000", 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950678+0000", 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950676+0000", 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950795+0000", 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118702 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950897+0000", 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010257200000000001 2024-09-16T06:49:05.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950920+0000", 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2784000000000001e-05 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.951018+0000", 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7981999999999998e-05 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.951060+0000", 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1353000000000002e-05 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957590+0000", 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065307220000000001 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957688+0000", 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7372000000000002e-05 2024-09-16T06:49:05.923 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 72/37 ECSubWrite(tid=81, reqid=client.4511.0:571, at_version=72'76, trim_to=0'0, roll_forward_to=72'72))", 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.013321+0000", 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.8184627549999997, 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0069873089999999997, 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013321+0000", 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013321+0000", 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013319+0000", 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013494+0000", 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017420099999999999 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013566+0000", 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2434000000000005e-05 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013579+0000", 2024-09-16T06:49:05.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3042e-05 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013631+0000", 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1907000000000003e-05 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013638+0000", 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6000000000000003e-06 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.020275+0000", 2024-09-16T06:49:05.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0066369819999999996 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.020308+0000", 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3531000000000001e-05 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/54 ECSubWrite(tid=184, reqid=osd.3.0:182, at_version=0'0, trim_to=0'0, roll_forward_to=73'67))", 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:05.928 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 4.962213223, 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058193519999999999, 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:05.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869583+0000", 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2876e-05 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869667+0000", 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4022000000000003e-05 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869677+0000", 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0681e-05 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869767+0000", 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9477999999999999e-05 2024-09-16T06:49:05.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869773+0000", 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0510000000000002e-06 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.875363+0000", 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055901900000000001 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.875390+0000", 2024-09-16T06:49:05.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6588999999999999e-05 2024-09-16T06:49:05.933 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.933 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.933 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.933 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:05.933 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:05.933 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:05.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:49:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:49:05.936 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:05.931+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:05.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:05.983 INFO:tasks.rados.rados.0.smithi117.stdout:266: done (0 left) 2024-09-16T06:49:05.983 INFO:tasks.rados.rados.0.smithi117.stdout:267: rollback oid 12 current snap is 28 2024-09-16T06:49:05.983 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 12 to 18 2024-09-16T06:49:05.983 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 11 current snap is 28 2024-09-16T06:49:05.983 INFO:tasks.rados.rados.0.smithi117.stdout:268: seq_num 121 ranges {0=491520,491520=802816,1294336=720896,2015232=32768} 2024-09-16T06:49:05.986 INFO:tasks.rados.rados.0.smithi117.stdout:268: writing smithi11731613-11 from 0 to 491520 tid 1 2024-09-16T06:49:05.992 INFO:tasks.rados.rados.0.smithi117.stdout:268: writing smithi11731613-11 from 491520 to 1294336 tid 2 2024-09-16T06:49:05.998 INFO:tasks.rados.rados.0.smithi117.stdout:268: writing smithi11731613-11 from 1294336 to 2015232 tid 3 2024-09-16T06:49:05.998 INFO:tasks.rados.rados.0.smithi117.stdout:268: writing smithi11731613-11 from 2015232 to 2048000 tid 4 2024-09-16T06:49:05.998 INFO:tasks.rados.rados.0.smithi117.stdout:269: read oid 49 snap -1 2024-09-16T06:49:05.998 INFO:tasks.rados.rados.0.smithi117.stdout:269: expect (ObjNum 103 snap 24 seq_num 103) 2024-09-16T06:49:05.998 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 23 current snap is 28 2024-09-16T06:49:05.999 INFO:tasks.rados.rados.0.smithi117.stdout:270: seq_num 122 ranges {1343488=507904,1851392=425984,2277376=442368,2719744=540672,3260416=147456} 2024-09-16T06:49:06.012 INFO:tasks.rados.rados.0.smithi117.stdout:270: writing smithi11731613-23 from 1343488 to 1851392 tid 1 2024-09-16T06:49:06.015 INFO:tasks.rados.rados.0.smithi117.stdout:270: writing smithi11731613-23 from 1851392 to 2277376 tid 2 2024-09-16T06:49:06.019 INFO:tasks.rados.rados.0.smithi117.stdout:270: writing smithi11731613-23 from 2277376 to 2719744 tid 3 2024-09-16T06:49:06.024 INFO:tasks.rados.rados.0.smithi117.stdout:270: writing smithi11731613-23 from 2719744 to 3260416 tid 4 2024-09-16T06:49:06.025 INFO:tasks.rados.rados.0.smithi117.stdout:270: writing smithi11731613-23 from 3260416 to 3407872 tid 5 2024-09-16T06:49:06.025 INFO:tasks.rados.rados.0.smithi117.stdout:271: rollback oid 20 current snap is 28 2024-09-16T06:49:06.025 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 20 to 26 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:267: finishing rollback tid 0 to smithi11731613-12 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:267: finishing rollback tid 1 to smithi11731613-12 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 12 v 100 (ObjNum 65 snap 0 seq_num 1678103472) dirty dne 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: finishing write tid 1 to smithi11731613-11 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: finishing write tid 2 to smithi11731613-11 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: finishing write tid 3 to smithi11731613-11 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: finishing write tid 4 to smithi11731613-11 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: finishing write tid 5 to smithi11731613-11 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: finishing write tid 6 to smithi11731613-11 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: oid 11 updating version 0 to 95 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: oid 11 updating version 95 to 96 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: oid 11 updating version 96 to 97 2024-09-16T06:49:06.026 INFO:tasks.rados.rados.0.smithi117.stdout:268: oid 11 version 97 is already newer than 93 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:268: oid 11 version 97 is already newer than 94 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 11 v 97 (ObjNum 121 snap 28 seq_num 121) dirty exists 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:268: left oid 11 (ObjNum 121 snap 28 seq_num 121) 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:267: done (4 left) 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:268: done (3 left) 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 29 current snap is 28 2024-09-16T06:49:06.027 INFO:tasks.rados.rados.0.smithi117.stdout:272: seq_num 123 ranges {475136=524288,999424=573440,1572864=688128,2260992=294912} 2024-09-16T06:49:06.032 INFO:tasks.rados.rados.0.smithi117.stdout:272: writing smithi11731613-29 from 475136 to 999424 tid 1 2024-09-16T06:49:06.035 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:06.031+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:06.036 INFO:tasks.rados.rados.0.smithi117.stdout:272: writing smithi11731613-29 from 999424 to 1572864 tid 2 2024-09-16T06:49:06.042 INFO:tasks.rados.rados.0.smithi117.stdout:272: writing smithi11731613-29 from 1572864 to 2260992 tid 3 2024-09-16T06:49:06.044 INFO:tasks.rados.rados.0.smithi117.stdout:272: writing smithi11731613-29 from 2260992 to 2555904 tid 4 2024-09-16T06:49:06.044 INFO:tasks.rados.rados.0.smithi117.stdout:273: snap_create 2024-09-16T06:49:06.046 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 1 to smithi11731613-23 2024-09-16T06:49:06.046 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 2 to smithi11731613-23 2024-09-16T06:49:06.046 INFO:tasks.rados.rados.0.smithi117.stdout:271: finishing rollback tid 1 to smithi11731613-20 2024-09-16T06:49:06.046 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 20 v 0 (ObjNum 33 snap 0 seq_num 1678103896) dirty dne 2024-09-16T06:49:06.046 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 3 to smithi11731613-23 2024-09-16T06:49:06.047 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 4 to smithi11731613-23 2024-09-16T06:49:06.047 INFO:tasks.rados.rados.0.smithi117.stdout:272: finishing write tid 1 to smithi11731613-29 2024-09-16T06:49:06.047 INFO:tasks.rados.rados.0.smithi117.stdout:272: finishing write tid 2 to smithi11731613-29 2024-09-16T06:49:06.052 INFO:tasks.rados.rados.0.smithi117.stdout:272: finishing write tid 3 to smithi11731613-29 2024-09-16T06:49:06.053 INFO:tasks.rados.rados.0.smithi117.stdout:269: expect (ObjNum 103 snap 24 seq_num 103) 2024-09-16T06:49:06.103 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:06.103 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:06.103 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:06.104 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:06.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:49:06.137 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:06.135+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:06.144 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:06.144 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:06.184 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:06.184 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:06.224 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:06.237 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:06.236+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:06.252 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 5 to smithi11731613-23 2024-09-16T06:49:06.252 INFO:tasks.rados.rados.0.smithi117.stdout:272: finishing write tid 4 to smithi11731613-29 2024-09-16T06:49:06.252 INFO:tasks.rados.rados.0.smithi117.stdout:272: finishing write tid 5 to smithi11731613-29 2024-09-16T06:49:06.252 INFO:tasks.rados.rados.0.smithi117.stdout:272: finishing write tid 6 to smithi11731613-29 2024-09-16T06:49:06.253 INFO:tasks.rados.rados.0.smithi117.stdout:272: oid 29 updating version 0 to 8 2024-09-16T06:49:06.253 INFO:tasks.rados.rados.0.smithi117.stdout:272: oid 29 version 8 is already newer than 6 2024-09-16T06:49:06.253 INFO:tasks.rados.rados.0.smithi117.stdout:272: oid 29 version 8 is already newer than 7 2024-09-16T06:49:06.253 INFO:tasks.rados.rados.0.smithi117.stdout:272: oid 29 version 8 is already newer than 4 2024-09-16T06:49:06.253 INFO:tasks.rados.rados.0.smithi117.stdout:272: oid 29 version 8 is already newer than 5 2024-09-16T06:49:06.253 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 29 v 8 (ObjNum 123 snap 28 seq_num 123) dirty exists 2024-09-16T06:49:06.254 INFO:tasks.rados.rados.0.smithi117.stdout:272: left oid 29 (ObjNum 123 snap 28 seq_num 123) 2024-09-16T06:49:06.261 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:06.262 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:06.266 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:06.266 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:06.266 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:06.266 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:06.266 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:06.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:49:06.293 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:06.294 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:06.318 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:49:06.318 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:49:06.339 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:06.336+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:06.343 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:06.343 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:06.368 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:06.391 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:06.391 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:06.421 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:06.421 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:46 3.0s0 3:043b7e8c:::smithi11731613-16:head [append 0~262144,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.917095+0000", 2024-09-16T06:49:06.424 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.510076908000002, 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014697692, 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 46 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917095+0000", 2024-09-16T06:49:06.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917095+0000", 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.917094+0000", 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918052+0000", 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095875799999999998 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918068+0000", 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5522000000000001e-05 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918077+0000", 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918141+0000", 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4046000000000003e-05 2024-09-16T06:49:06.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918300+0000", 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159481 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.919368+0000", 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001067337 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924493+0000", 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051250860000000001 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.931676+0000", 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071828500000000002 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.931793+0000", 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117336 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.430 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:47 3.0s0 3:043b7e8c:::smithi11731613-16:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918091+0000", 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.509081684999998, 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.01424603, 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 47 2024-09-16T06:49:06.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918091+0000", 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918091+0000", 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918090+0000", 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918096+0000", 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3450000000000002e-06 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918105+0000", 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.149e-06 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918111+0000", 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1299999999999998e-06 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.919700+0000", 2024-09-16T06:49:06.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001588423 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.919759+0000", 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8881000000000001e-05 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920270+0000", 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051165300000000004 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.924494+0000", 2024-09-16T06:49:06.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042238670000000001 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.932107+0000", 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076127310000000002 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.932337+0000", 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000229788 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.436 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:48 3.0s0 3:043b7e8c:::smithi11731613-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.508317452, 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.016720260000000001, 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.437 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 48 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918854+0000", 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918860+0000", 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3960000000000003e-06 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918873+0000", 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2485e-05 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918923+0000", 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0386999999999997e-05 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920454+0000", 2024-09-16T06:49:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015303949999999999 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920536+0000", 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2573999999999999e-05 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.933186+0000", 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.012650051000000001 2024-09-16T06:49:06.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.933335+0000", 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000148689 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935575+0000", 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022402540000000001 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:59 3.0s0 3:0900ee20:::smithi11731613-20:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.487792483, 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.069099573999999997, 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 59 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.443 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939379+0000", 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939383+0000", 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9040000000000002e-06 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939390+0000", 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:49:06.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939394+0000", 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9380000000000002e-06 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943863+0000", 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044688669999999996 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943950+0000", 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7173e-05 2024-09-16T06:49:06.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944394+0000", 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044454100000000001 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944887+0000", 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000492156 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008423+0000", 2024-09-16T06:49:06.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.063536255 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008479+0000", 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6462999999999998e-05 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:60 3.0s0 3:0900ee20:::smithi11731613-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.448 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.487730524, 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.076005592999999996, 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 60 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:49:06.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939441+0000", 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939445+0000", 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7340000000000002e-06 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939451+0000", 2024-09-16T06:49:06.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.241e-06 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939457+0000", 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4580000000000001e-06 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944649+0000", 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051924110000000001 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944778+0000", 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012887900000000001 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008904+0000", 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.064125946000000003 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.009068+0000", 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016377000000000001 2024-09-16T06:49:06.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.015447+0000", 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063797940000000003 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:101 3.9s0 3:92c79891:::smithi11731613-34:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.454 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.424889841000002, 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011878148999999999, 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 101 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002282+0000", 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002286+0000", 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4780000000000002e-06 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002293+0000", 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0879999999999997e-06 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002299+0000", 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5509999999999999e-06 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.006252+0000", 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003953397 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.006430+0000", 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017774800000000001 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007019+0000", 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00058882300000000004 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.009099+0000", 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020800160000000001 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014089+0000", 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049907900000000002 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014161+0000", 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1157999999999999e-05 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:102 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.424807311000002, 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.046755945, 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 102 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002364+0000", 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002370+0000", 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5459999999999999e-06 2024-09-16T06:49:06.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002385+0000", 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5724000000000001e-05 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002394+0000", 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3280000000000006e-06 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007411+0000", 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050172059999999997 2024-09-16T06:49:06.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007528+0000", 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117267 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014384+0000", 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006855809 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014457+0000", 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2799999999999994e-05 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049121+0000", 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.034664014 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:127 3.0s0 3:03b32ae3:::smithi11731613-43:head [append 0~720896,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.039739+0000", 2024-09-16T06:49:06.465 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.387432867999998, 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011181432999999999, 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 127 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.039739+0000", 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.039739+0000", 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.039738+0000", 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046439+0000", 2024-09-16T06:49:06.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006700981 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046452+0000", 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2411e-05 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046459+0000", 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7859999999999997e-06 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046623+0000", 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012280700000000001 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.047810+0000", 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011871679999999999 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050744+0000", 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002934 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050881+0000", 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013747999999999999 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050921+0000", 2024-09-16T06:49:06.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9391999999999998e-05 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:128 3.0s0 3:03b32ae3:::smithi11731613-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.380699714999999, 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.031368511000000002, 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.471 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 128 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046472+0000", 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046476+0000", 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.578e-06 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046485+0000", 2024-09-16T06:49:06.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2289999999999992e-06 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046491+0000", 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6319999999999996e-06 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048421+0000", 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019297240000000001 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048515+0000", 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4073999999999994e-05 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048890+0000", 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000375306 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050760+0000", 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018703350000000001 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077776+0000", 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.027015092000000001 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077841+0000", 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5504000000000002e-05 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:129 3.0s0 3:03b32ae3:::smithi11731613-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.380672695000001, 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.036392558999999998, 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 129 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046499+0000", 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:06.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046502+0000", 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8200000000000001e-06 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046507+0000", 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3510000000000001e-06 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046512+0000", 2024-09-16T06:49:06.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.464e-06 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049021+0000", 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025094789999999998 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049099+0000", 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7998999999999996e-05 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077886+0000", 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.028786875 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077945+0000", 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8941999999999999e-05 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.082892+0000", 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049470909999999998 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:157 3.0s0 3:043b7e8c:::smithi11731613-16:head [read 0~262144,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.344023292000003, 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14327616900000001, 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 157 2024-09-16T06:49:06.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083148+0000", 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083152+0000", 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1920000000000001e-06 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083158+0000", 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1959999999999996e-06 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083163+0000", 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2909999999999998e-06 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084618+0000", 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0014551430000000001 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084738+0000", 2024-09-16T06:49:06.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119295 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.226425+0000", 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14168755499999999 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:292 3.0s0 3:03b32ae3:::smithi11731613-43:head [append 0~0] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.323014972999999, 2024-09-16T06:49:06.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011795597, 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 292 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104157+0000", 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104160+0000", 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5820000000000002e-06 2024-09-16T06:49:06.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104169+0000", 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4260000000000003e-06 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.104175+0000", 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6710000000000002e-06 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.107609+0000", 2024-09-16T06:49:06.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003433204 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.108605+0000", 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099621299999999992 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.109122+0000", 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051705200000000005 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.114736+0000", 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005614165 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.115910+0000", 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011743229999999999 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:30.115953+0000", 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2521000000000002e-05 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.491 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:358 3.0s0 3:03b32ae3:::smithi11731613-43:head [delete] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:34.845297+0000", 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 31.581874769999999, 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014407524, 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 358 2024-09-16T06:49:06.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845297+0000", 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845297+0000", 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845294+0000", 2024-09-16T06:49:06.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845313+0000", 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9170000000000001e-05 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845350+0000", 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6766e-05 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845368+0000", 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.772e-05 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845475+0000", 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010686199999999999 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op must wait for map", 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.845491+0000", 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.694e-05 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.851263+0000", 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057716750000000004 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.851521+0000", 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025770699999999998 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.853488+0000", 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001967594 2024-09-16T06:49:06.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.856388+0000", 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002899502 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.859532+0000", 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031442229999999998 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:34.859705+0000", 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017284200000000001 2024-09-16T06:49:06.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.cs3 49/37 ECSubWrite(tid=98, reqid=osd.11.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=49'26))", 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.571935534000001, 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14832383499999999, 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855241+0000", 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5889999999999996e-06 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002739+0000", 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14749786200000001 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002747+0000", 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5390000000000002e-06 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002811+0000", 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4498999999999994e-05 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002824+0000", 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2883e-05 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.003527+0000", 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070348000000000003 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.003561+0000", 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3121999999999999e-05 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:574 3.0s0 3:03b32ae3:::smithi11731613-43:head [copy-from ver 8] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.008959+0000", 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.418213325, 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014062869, 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.502 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 574 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.008959+0000", 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.008959+0000", 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.008955+0000", 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.008974+0000", 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8535000000000001e-05 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009016+0000", 2024-09-16T06:49:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1794e-05 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009039+0000", 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.368e-05 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009098+0000", 2024-09-16T06:49:06.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8579000000000003e-05 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009164+0000", 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5767999999999998e-05 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.017392+0000", 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.008228183 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.020297+0000", 2024-09-16T06:49:06.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029055639999999998 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.022979+0000", 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026812759999999998 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.023022+0000", 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3099999999999997e-05 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:579 3.9s0 3:92c79891:::smithi11731613-34:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.063604+0000", 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.3635684809999997, 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011949249, 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 579 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.508 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063604+0000", 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063604+0000", 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063602+0000", 2024-09-16T06:49:06.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063615+0000", 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2466000000000001e-05 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063635+0000", 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0197e-05 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063649+0000", 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4250000000000001e-05 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.070076+0000", 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064266280000000002 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.070438+0000", 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036222400000000002 2024-09-16T06:49:06.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.071377+0000", 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00093877200000000004 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.072696+0000", 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001318559 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.075319+0000", 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026230540000000001 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.075553+0000", 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000234479 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:580 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.063662+0000", 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 6.363510089, 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015297809000000001, 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 580 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063662+0000", 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063662+0000", 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063661+0000", 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063668+0000", 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7900000000000002e-06 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063684+0000", 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6342999999999999e-05 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.063697+0000", 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2384e-05 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.073799+0000", 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010102457 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.073944+0000", 2024-09-16T06:49:06.517 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145186 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.076417+0000", 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024724090000000001 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.076611+0000", 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000194465 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.078960+0000", 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023489549999999998 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:674 3.9s0 3:92c79891:::smithi11731613-34:head [setxattr _header (57) in=7b] snapc 2a=[2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:03.908895+0000", 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 2.5182772230000001, 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.37177128500000001, 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 674 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.520 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908895+0000", 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908895+0000", 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908894+0000", 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908899+0000", 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4129999999999999e-06 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908905+0000", 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.815e-06 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.522 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908911+0000", 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9800000000000003e-06 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910067+0000", 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011550779999999999 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910153+0000", 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6929999999999996e-05 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.523 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910552+0000", 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039885300000000001 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.912889+0000", 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002336893 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280537+0000", 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.367647797 2024-09-16T06:49:06.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280666+0000", 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012930999999999999 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:675 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e76)", 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:03.908918+0000", 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 2.518254706, 2024-09-16T06:49:06.525 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.37382299800000002, 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 675 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908918+0000", 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908918+0000", 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908917+0000", 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.527 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908920+0000", 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9289999999999998e-06 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908927+0000", 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4760000000000003e-06 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908931+0000", 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1799999999999998e-06 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910904+0000", 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019736509999999999 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911021+0000", 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011655900000000001 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.281221+0000", 2024-09-16T06:49:06.529 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.37019976599999999 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.281359+0000", 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138305 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.282741+0000", 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013815870000000001 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 76/32 ECSubWriteReply(tid=92, last_complete=76'28, committed=1, applied=1))", 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 2.515629202, 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.36963403, 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:06.531 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911544+0000", 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2869999999999999e-06 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280309+0000", 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.36876511000000001 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280325+0000", 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5860000000000001e-05 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280408+0000", 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2886999999999994e-05 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:06.533 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.281177+0000", 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076903900000000001 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:06.534 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:06.536 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:49:06.536 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:06.537 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:06.537 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:06.538 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:06.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:06.516+0000 7fb14ffff640 0 -- 172.21.15.100:0/1392961634 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7fb15821e8b0 msgr2=0x7fb158220cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:06.539 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:06.531+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:06.636 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:06.635+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:06.664 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:06.664 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:06.664 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 6 to smithi11731613-23 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: finishing write tid 7 to smithi11731613-23 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: oid 23 updating version 0 to 99 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: oid 23 updating version 99 to 103 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: oid 23 updating version 103 to 104 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: oid 23 version 104 is already newer than 101 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: oid 23 version 104 is already newer than 102 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: oid 23 version 104 is already newer than 100 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 23 v 104 (ObjNum 122 snap 28 seq_num 122) dirty exists 2024-09-16T06:49:06.667 INFO:tasks.rados.rados.0.smithi117.stdout:270: left oid 23 (ObjNum 122 snap 28 seq_num 122) 2024-09-16T06:49:06.678 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:49:06.737 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:06.738+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:06.748 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:06.748 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:06.787 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:06.787 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:06.804 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:06.804 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:06.818 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:49:06.819 INFO:tasks.rados.rados.0.smithi117.stdout:269: done (4 left) 2024-09-16T06:49:06.819 INFO:tasks.rados.rados.0.smithi117.stdout:270: done (3 left) 2024-09-16T06:49:06.819 INFO:tasks.rados.rados.0.smithi117.stdout:271: done (2 left) 2024-09-16T06:49:06.819 INFO:tasks.rados.rados.0.smithi117.stdout:272: done (1 left) 2024-09-16T06:49:06.820 INFO:tasks.rados.rados.0.smithi117.stdout:273: done (0 left) 2024-09-16T06:49:06.820 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 20 current snap is 29 2024-09-16T06:49:06.820 INFO:tasks.rados.rados.0.smithi117.stdout:274: seq_num 124 ranges {0=540672,540672=720896,1261568=786432,2048000=49152} 2024-09-16T06:49:06.824 INFO:tasks.rados.rados.0.smithi117.stdout:274: writing smithi11731613-20 from 0 to 540672 tid 1 2024-09-16T06:49:06.829 INFO:tasks.rados.rados.0.smithi117.stdout:274: writing smithi11731613-20 from 540672 to 1261568 tid 2 2024-09-16T06:49:06.836 INFO:tasks.rados.rados.0.smithi117.stdout:274: writing smithi11731613-20 from 1261568 to 2048000 tid 3 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:274: writing smithi11731613-20 from 2048000 to 2097152 tid 4 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:275: rollback oid 21 current snap is 29 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 21 to 13 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:274: finishing write tid 1 to smithi11731613-20 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:276: read oid 28 snap -1 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:276: expect deleted 2024-09-16T06:49:06.837 INFO:tasks.rados.rados.0.smithi117.stdout:277: snap_remove snap 22 2024-09-16T06:49:06.861 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:06.862+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:06.939 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:06.936+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:06.985 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:06.985 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:06.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:49:07.048 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:07.050+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:07.121 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:07.121 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:07.136 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T06:49:07.140 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:07.135+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:07.241 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:07.244+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:07.296 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:07.297 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:07.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T06:49:07.342 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:07.339+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:07.434 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:07.435 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:07.442 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:07.442+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:07.449 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:49:07.543 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:07.539+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:07.615 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:07.615 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:07.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:49:07.644 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:07.643+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:07.745 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:07.746+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:49:07.753 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:07.753 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:07.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:277: done (3 left) 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:278: rmattr oid 43 current snap is 29 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:278: done (3 left) 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:279: delete oid 4 current snap is 29 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:275: finishing rollback tid 0 to smithi11731613-21 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:275: finishing rollback tid 1 to smithi11731613-21 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 21 v 101 (ObjNum 49 snap 0 seq_num 12) dirty dne 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:274: finishing write tid 2 to smithi11731613-20 2024-09-16T06:49:07.829 INFO:tasks.rados.rados.0.smithi117.stdout:274: finishing write tid 3 to smithi11731613-20 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: finishing write tid 4 to smithi11731613-20 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: finishing write tid 5 to smithi11731613-20 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: finishing write tid 6 to smithi11731613-20 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: oid 20 updating version 0 to 59 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: oid 20 version 59 is already newer than 58 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: oid 20 updating version 59 to 62 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: oid 20 version 62 is already newer than 60 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: oid 20 version 62 is already newer than 61 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 20 v 62 (ObjNum 124 snap 29 seq_num 124) dirty exists 2024-09-16T06:49:07.830 INFO:tasks.rados.rados.0.smithi117.stdout:274: left oid 20 (ObjNum 124 snap 29 seq_num 124) 2024-09-16T06:49:07.856 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:07.858+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:49:07.858 INFO:tasks.rados.rados.0.smithi117.stdout:274: done (3 left) 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:275: done (2 left) 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:276: done (1 left) 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:279: done (0 left) 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:280: read oid 6 snap -1 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:280: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:281: copy_from oid 39 from oid 31 current snap is 29 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:282: rmattr oid 13 current snap is 29 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:282: done (2 left) 2024-09-16T06:49:07.859 INFO:tasks.rados.rados.0.smithi117.stdout:283: snap_create 2024-09-16T06:49:07.863 INFO:tasks.rados.rados.0.smithi117.stdout:280: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:07.868 INFO:tasks.rados.rados.0.smithi117.stdout:281: finishing copy_from to smithi11731613-39 2024-09-16T06:49:07.868 INFO:tasks.rados.rados.0.smithi117.stdout:281: got expected ENOENT (src dne) 2024-09-16T06:49:07.869 INFO:tasks.rados.rados.0.smithi117.stdout:281: finishing copy_from racing read to smithi11731613-39 2024-09-16T06:49:07.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:49:07.947 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:07.947+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:07.947 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:07.947 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:07.963 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:49:08.048 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:08.043+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:08.126 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:08.126 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:08.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:49:08.149 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:08.143+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:08.250 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:08.251+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:08.263 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:08.264 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:08.278 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:49:08.351 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:08.347+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:08.399 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:08.399 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:08.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:49:08.460 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:08.459+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:08.544 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:08.544 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:08.553 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:08.551+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:08.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:49:08.655 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:08.655+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:08.720 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:08.721 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:08.734 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:49:08.755 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:08.754+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:49:08.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:08.829 INFO:tasks.rados.rados.0.smithi117.stdout:280: done (2 left) 2024-09-16T06:49:08.829 INFO:tasks.rados.rados.0.smithi117.stdout:281: done (1 left) 2024-09-16T06:49:08.829 INFO:tasks.rados.rados.0.smithi117.stdout:283: done (0 left) 2024-09-16T06:49:08.829 INFO:tasks.rados.rados.0.smithi117.stdout:284: rollback oid 3 current snap is 30 2024-09-16T06:49:08.829 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 3 to 14 2024-09-16T06:49:08.829 INFO:tasks.rados.rados.0.smithi117.stdout:285: copy_from oid 25 from oid 11 current snap is 30 2024-09-16T06:49:08.830 INFO:tasks.rados.rados.0.smithi117.stdout:286: snap_remove snap 23 2024-09-16T06:49:08.830 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:49:08.856 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:08.858+0000 7f6a78b66640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26169) UID: 0 2024-09-16T06:49:08.864 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:08.864 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:08.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:49:08.957 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:08.955+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:08.981 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:08.982 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:09.000 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:09.006 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:09.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:49:09.031 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:09.031 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:09.058 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:09.059 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:09.059 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:09.055+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:09.085 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:09.085 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:09.114 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-16T06:49:09.114 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T06:49:09.136 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:09.136 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:09.141 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:09.141 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:09.151 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:49:09.161 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:09.160+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:09.171 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:09.196 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:09.196 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:09.226 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:09.226 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:09.256 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:09.262 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:09.259+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:09.268 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.617477925000003, 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:49:09.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803677747000002, 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:49:09.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:49:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:09.284 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803549195000002, 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:49:09.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:49:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:09.291 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803511868000001, 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:09.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:49:09.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803376870000001, 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:49:09.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803344709000001, 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:49:09.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:49:09.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:49:09.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:49:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803316277999997, 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:49:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:49:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:49:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.803209359, 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:49:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:49:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.109702847999998, 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:49:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:49:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.109672600000003, 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:49:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:49:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:49:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.109643165000001, 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:49:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:49:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:49:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:49:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.109532246000001, 2024-09-16T06:49:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:49:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:49:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:49:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:77 3.8s0 3:119b2380:::smithi11731613-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.296362719999998, 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055956466000000003, 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 77 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971691+0000", 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971701+0000", 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8519999999999999e-06 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971715+0000", 2024-09-16T06:49:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4386000000000001e-05 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971725+0000", 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0360000000000001e-05 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.974948+0000", 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003222759 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975027+0000", 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8497000000000004e-05 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975520+0000", 2024-09-16T06:49:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049320500000000003 2024-09-16T06:49:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.977378+0000", 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018576230000000001 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027491+0000", 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050112973999999998 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027649+0000", 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015816800000000001 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:78 3.8s0 3:119b2380:::smithi11731613-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.296267706000002, 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059293289999999998, 2024-09-16T06:49:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971786+0000", 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971793+0000", 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971809+0000", 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5225000000000001e-05 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971820+0000", 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0922e-05 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975891+0000", 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040709759999999996 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.976067+0000", 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017645900000000001 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027711+0000", 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051643840000000003 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027855+0000", 2024-09-16T06:49:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014441399999999999 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031081+0000", 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032252359999999998 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 35/32 ECSubWriteReply(tid=162, last_complete=35'4, committed=1, applied=1))", 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.235302243, 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48350030599999999, 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032752+0000", 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032763+0000", 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1807000000000001e-05 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516055+0000", 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48329140799999998 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516068+0000", 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2782999999999999e-05 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516198+0000", 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129949 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516253+0000", 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5390000000000003e-05 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=17, reqid=client.4511.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.212205978999997, 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.51123769299999999, 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055848+0000", 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055863+0000", 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.464e-05 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565781+0000", 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50991880999999994 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565796+0000", 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4263e-05 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565932+0000", 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013609900000000001 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565949+0000", 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7229000000000001e-05 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567019+0000", 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001070463 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567087+0000", 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7229000000000003e-05 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.7s2 40/37 ECSubRead(tid=11, to_read={3:e60b6e1b:::smithi11731613-14:head=0,4096,0}, subchunks={3:e60b6e1b:::smithi11731613-14:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.200524584, 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.375282063, 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067529+0000", 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067547+0000", 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8187000000000001e-05 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442125+0000", 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.37457846099999997 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442142+0000", 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6843000000000001e-05 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442313+0000", 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017038200000000001 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442812+0000", 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049983999999999998 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.426491339, 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030302045, 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841562+0000", 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841613+0000", 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1739e-05 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841692+0000", 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8974000000000007e-05 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841718+0000", 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5452999999999998e-05 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847063+0000", 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053447629999999998 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847087+0000", 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4553999999999999e-05 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871791+0000", 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024703433 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871866+0000", 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5133e-05 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s4 71/32 ECSubWrite(tid=63, reqid=osd.12.0:63, at_version=71'48, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.424827458999999, 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623895, 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843226+0000", 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843302+0000", 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5922999999999998e-05 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843397+0000", 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5536000000000001e-05 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843419+0000", 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2240000000000001e-05 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847145+0000", 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037257950000000001 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847167+0000", 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1590000000000002e-05 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871800+0000", 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024633057 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871851+0000", 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1647000000000002e-05 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:564 3.8s0 3:1cef29f0:::smithi11731613-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.2925724550000002, 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020010844, 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 564 2024-09-16T06:49:09.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975481+0000", 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975500+0000", 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975518+0000", 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975531+0000", 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3207000000000001e-05 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.983862+0000", 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083305979999999998 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.984001+0000", 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000139629 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990097+0000", 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060955419999999998 2024-09-16T06:49:09.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990219+0000", 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122349 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.995493+0000", 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052739930000000003 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.409 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:09.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:09.410 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:09.411 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:09.412 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:09.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:49:09.412 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:09.413 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:09.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:09.376+0000 7fcd02141640 0 -- 172.21.15.100:0/2857699365 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7fccfc2217e0 msgr2=0x7fccfc223bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:09.414 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:09.411+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:09.511 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:09.507+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:09.540 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:09.540 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:09.540 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:09.540 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:09.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:49:09.609 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:09.609 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:09.611 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:09.607+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:09.676 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:09.676 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:09.677 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:09.677 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:09.677 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:09.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:49:09.711 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:09.707+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:09.810 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:09.810 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:09.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:09.810 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:09.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.910698564999997, 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022146409999999998, 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897264+0000", 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897309+0000", 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5123000000000001e-05 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913004+0000", 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015694784999999999 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913046+0000", 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2945999999999998e-05 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914353+0000", 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306104 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914377+0000", 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4562999999999998e-05 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919353+0000", 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049759360000000002 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919412+0000", 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9327000000000003e-05 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:22 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.908423022000001, 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023053033000000001, 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 22 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899530+0000", 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901021+0000", 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014909019999999999 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901056+0000", 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5136999999999999e-05 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901077+0000", 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1622999999999999e-05 2024-09-16T06:49:09.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901274+0000", 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019618500000000001 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901606+0000", 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033268099999999998 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903794+0000", 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021876339999999999 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907586+0000", 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003792489 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922327+0000", 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014741124 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922595+0000", 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026710500000000001 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:23 3.3s0 3:ce1274e9:::smithi11731613-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.906859820000001, 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022719541999999999, 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:09.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901103+0000", 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901115+0000", 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1722999999999999e-05 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901134+0000", 2024-09-16T06:49:09.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9213000000000001e-05 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901149+0000", 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.523e-05 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905252+0000", 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041033170000000004 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905520+0000", 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026748400000000002 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.906744+0000", 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012242209999999999 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923321+0000", 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016576822000000001 2024-09-16T06:49:09.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923623+0000", 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249399999999999 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923824+0000", 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000200955 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:24 3.3s0 3:ce1274e9:::smithi11731613-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.906800994000001, 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033126111999999999, 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901162+0000", 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901169+0000", 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901187+0000", 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8320000000000001e-05 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901203+0000", 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5872999999999999e-05 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907294+0000", 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006090949 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907512+0000", 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021778499999999999 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924007+0000", 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016495668000000002 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924218+0000", 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021115199999999999 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934290+0000", 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010071302000000001 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:31 3.3s0 3:c7654c4c:::smithi11731613-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.906748671000003, 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026931531000000002, 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901215+0000", 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902241+0000", 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010258190000000001 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902275+0000", 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4037e-05 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902312+0000", 2024-09-16T06:49:09.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.718e-05 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053663829999999997 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.908030+0000", 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035200800000000001 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912004+0000", 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039736770000000001 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923909+0000", 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011905429 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927953+0000", 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040441499999999998 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928147+0000", 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194005 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:32 3.3s0 3:c7654c4c:::smithi11731613-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.905632613000002, 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028234555000000001, 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 32 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:09.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902330+0000", 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902341+0000", 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1279e-05 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902360+0000", 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902374+0000", 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3917000000000001e-05 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917230+0000", 2024-09-16T06:49:09.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014855607 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917486+0000", 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025648799999999997 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918644+0000", 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011572749999999999 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928363+0000", 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097195389999999993 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930283+0000", 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919505 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930566+0000", 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028369099999999998 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:33 3.3s0 3:c7654c4c:::smithi11731613-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.905577923000003, 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037765197, 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 33 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:09.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902386+0000", 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902392+0000", 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893999999999998e-05 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902429+0000", 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8689999999999999e-05 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934639+0000", 2024-09-16T06:49:09.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032210404999999998 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934843+0000", 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020376199999999999 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.940152+0000", 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053090200000000002 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=4, reqid=client.4511.0:44, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.890661237000003, 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788766000000001, 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917302+0000", 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917349+0000", 2024-09-16T06:49:09.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7796000000000001e-05 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941504+0000", 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024154637 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941525+0000", 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0608999999999999e-05 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941640+0000", 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011531000000000001 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941662+0000", 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2328999999999999e-05 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943041+0000", 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001379047 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943092+0000", 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0834000000000002e-05 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:68 3.3s0 3:cbbd057c:::smithi11731613-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.853939623000002, 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045964098000000002, 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 68 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954024+0000", 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954031+0000", 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954046+0000", 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954058+0000", 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1938e-05 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956544+0000", 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024857949999999998 2024-09-16T06:49:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956742+0000", 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019819 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.957650+0000", 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090843 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959725+0000", 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074256 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999914+0000", 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040189895000000003 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999989+0000", 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4527e-05 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:69 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.853895137999999, 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048288992000000003, 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 69 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954068+0000", 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954074+0000", 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954088+0000", 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4866e-05 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954100+0000", 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1328999999999999e-05 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959103+0000", 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035749999999997 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959283+0000", 2024-09-16T06:49:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017953899999999999 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000081+0000", 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040797893000000002 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000261+0000", 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180304 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002358+0000", 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002097392 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:98 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.812471272000003, 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.086611296000000004, 2024-09-16T06:49:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995492+0000", 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995501+0000", 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8720000000000008e-06 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995517+0000", 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6650000000000002e-05 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995529+0000", 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1447999999999999e-05 2024-09-16T06:49:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.997874+0000", 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023456250000000001 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998037+0000", 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016212600000000001 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998796+0000", 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075897099999999999 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002377+0000", 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00358195 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081972+0000", 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.079594867 2024-09-16T06:49:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082105+0000", 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013220899999999999 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:99 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.812404209, 2024-09-16T06:49:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.087893389000000002, 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995559+0000", 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995565+0000", 2024-09-16T06:49:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2659999999999998e-06 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995581+0000", 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.611e-05 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995594+0000", 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2269000000000001e-05 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999116+0000", 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003522238 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999227+0000", 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111483 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082346+0000", 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.083118743999999994 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082416+0000", 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9993000000000006e-05 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.083454+0000", 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00103751 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:158 3.3s0 3:c7654c4c:::smithi11731613-11:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.728178399000001, 2024-09-16T06:49:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41982487699999999, 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079784+0000", 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079794+0000", 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6260000000000007e-06 2024-09-16T06:49:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079810+0000", 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6631000000000001e-05 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079821+0000", 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0849000000000001e-05 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079917+0000", 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5730999999999995e-05 2024-09-16T06:49:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.080126+0000", 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020895399999999999 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081069+0000", 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094364700000000004 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082604+0000", 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001534863 2024-09-16T06:49:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499525+0000", 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41692029400000002 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499611+0000", 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-05 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:226 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.852297598, 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027980046000000001, 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 226 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955665+0000", 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955672+0000", 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000002e-06 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955690+0000", 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8179999999999999e-05 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955704+0000", 2024-09-16T06:49:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.396e-05 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956020+0000", 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000316101 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956038+0000", 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8425999999999999e-05 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974108+0000", 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018069776999999999 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974338+0000", 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022928799999999999 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975586+0000", 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012484639999999999 2024-09-16T06:49:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.979590+0000", 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040043939999999997 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983481+0000", 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038905480000000002 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983647+0000", 2024-09-16T06:49:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166045 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:348 3.3s0 3:c742dae1:::smithi11731613-33:head [append 0~622592] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.967275241999999, 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020152403999999999, 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 348 2024-09-16T06:49:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840687+0000", 2024-09-16T06:49:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841715+0000", 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010285260000000001 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841743+0000", 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7990000000000001e-05 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841758+0000", 2024-09-16T06:49:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4686e-05 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842034+0000", 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027588800000000001 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842198+0000", 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000164599 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.843642+0000", 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014432760000000001 2024-09-16T06:49:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.847983+0000", 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043409349999999998 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860716+0000", 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012732902000000001 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860842+0000", 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012617499999999999 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:350 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (57) in=7b] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.966040374000002, 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019634848, 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 350 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841923+0000", 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841929+0000", 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6060000000000002e-06 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841938+0000", 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4970000000000007e-06 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841946+0000", 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9130000000000003e-06 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.848163+0000", 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062167339999999998 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.848319+0000", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015559300000000001 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849250+0000", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093117299999999998 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.853284+0000", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040342399999999997 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.861427+0000", 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081432659999999997 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.861559+0000", 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013169599999999999 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:351 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.966011217999998, 2024-09-16T06:49:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023956603, 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 351 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841957+0000", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0199999999999996e-06 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841966+0000", 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-06 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841974+0000", 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.34e-06 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849799+0000", 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078258089999999995 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.850139+0000", 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033906799999999999 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862313+0000", 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012174318999999999 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862630+0000", 2024-09-16T06:49:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031676099999999998 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.865910+0000", 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032802859999999999 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 47/32 ECSubWrite(tid=68, reqid=client.4511.0:361, at_version=0'0, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.936447125999997, 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50083384399999997, 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871516+0000", 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871541+0000", 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5782000000000002e-05 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370610+0000", 2024-09-16T06:49:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.49906899100000002 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370625+0000", 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.429e-05 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370744+0000", 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119746 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370773+0000", 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8807e-05 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372302+0000", 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015286519999999999 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372351+0000", 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9348999999999998e-05 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 49/32 ECSubWrite(tid=70, reqid=osd.3.0:70, at_version=49'51, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.952759954000001, 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023397583, 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855204+0000", 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967296 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855221+0000", 2024-09-16T06:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877682+0000", 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022460635999999999 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877690+0000", 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5369999999999997e-06 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877755+0000", 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4584000000000004e-05 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877766+0000", 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0968000000000001e-05 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878577+0000", 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000811029 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878602+0000", 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5191e-05 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 73/32 ECSubWriteReply(tid=108, last_complete=73'90, committed=1, applied=1))", 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.9382046039999992, 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071784013999999993, 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869759+0000", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869765+0000", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7590000000000003e-06 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941447+0000", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071682318999999994 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941458+0000", 2024-09-16T06:49:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1099e-05 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941509+0000", 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1201e-05 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941544+0000", 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4538e-05 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:09.914 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:09.914 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 13] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:49:09.914 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:49:09.914 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:49:09.914 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T06:49:09.915 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:49:09.916 INFO:tasks.rados.rados.0.smithi117.stdout:286: done (2 left) 2024-09-16T06:49:09.916 INFO:tasks.rados.rados.0.smithi117.stdout:287: read oid 34 snap -1 2024-09-16T06:49:09.916 INFO:tasks.rados.rados.0.smithi117.stdout:287: expect (ObjNum 117 snap 27 seq_num 117) 2024-09-16T06:49:09.916 INFO:tasks.rados.rados.0.smithi117.stdout:288: read oid 13 snap 24 2024-09-16T06:49:09.916 INFO:tasks.rados.rados.0.smithi117.stdout:288: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:49:09.916 INFO:tasks.rados.rados.0.smithi117.stdout:289: snap_remove snap 28 2024-09-16T06:49:09.917 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:49:09.917 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:09.911+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:09.987 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T06:49:09.987 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-16T06:49:09.988 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:49:09.988 DEBUG:teuthology.orchestra.run.smithi107:> type ceph-objectstore-tool 2024-09-16T06:49:09.993 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T06:49:09.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-16T06:49:10.016 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:10.011+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:10.044 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:10.044 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:10.044 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:10.044 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:10.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:49:10.117 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:10.115+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:10.218 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:10.214+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:10.222 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:10.222 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:10.222 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:10.236 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:49:10.319 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:10.315+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:10.402 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:10.402 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:10.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:10.402 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:10.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.402 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds1 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.501527697, 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018662743999999998, 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898052+0000", 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898050+0000", 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898094+0000", 2024-09-16T06:49:10.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3769000000000001e-05 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915572+0000", 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017478235000000002 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915584+0000", 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1996e-05 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915648+0000", 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3551000000000002e-05 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915661+0000", 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2897e-05 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916697+0000", 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001036653 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916715+0000", 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7376000000000001e-05 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds1 35/32 ECSubWrite(tid=5, reqid=client.4511.0:50, at_version=35'4, trim_to=0'0, roll_forward_to=34'2))", 2024-09-16T06:49:10.407 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.470760888999997, 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023812890999999999, 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928819+0000", 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928818+0000", 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.928835+0000", 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7011999999999999e-05 2024-09-16T06:49:10.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949938+0000", 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.02110269 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.949947+0000", 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0010000000000007e-06 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950024+0000", 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6520000000000006e-05 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.950032+0000", 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6750000000000008e-06 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952592+0000", 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002559296 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952632+0000", 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0243000000000001e-05 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:74 3.as0 3:5d679340:::smithi11731613-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.437303475, 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026692272999999999, 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.412 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 74 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962275+0000", 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962280+0000", 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1300000000000003e-06 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962286+0000", 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4529999999999999e-06 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962292+0000", 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5849999999999999e-06 2024-09-16T06:49:10.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964248+0000", 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001956667 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964344+0000", 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6174000000000004e-05 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964765+0000", 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042082000000000003 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965862+0000", 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0010968289999999999 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988875+0000", 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023013051 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988969+0000", 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3350000000000006e-05 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:75 3.as0 3:5d679340:::smithi11731613-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.437275163000002, 2024-09-16T06:49:10.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033077179999999998, 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 75 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:10.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962304+0000", 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962308+0000", 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0600000000000001e-06 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962314+0000", 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.088e-06 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962319+0000", 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1120000000000004e-06 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964978+0000", 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026582540000000001 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965065+0000", 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7108999999999996e-05 2024-09-16T06:49:10.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989270+0000", 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024205503 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989359+0000", 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9204999999999997e-05 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995382+0000", 2024-09-16T06:49:10.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060223200000000003 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=11, last_complete=35'6, committed=1, applied=1))", 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.433539504999999, 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022946498999999999, 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966040+0000", 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966039+0000", 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.966044+0000", 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6959999999999996e-06 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988553+0000", 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022508866999999998 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988563+0000", 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0373e-05 2024-09-16T06:49:10.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988804+0000", 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000240289 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988987+0000", 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018304199999999999 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.427 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:95 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.407390587000002, 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042547695000000003, 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.428 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 95 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992188+0000", 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992194+0000", 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8760000000000002e-06 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.430 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992204+0000", 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.011e-05 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992212+0000", 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1890000000000001e-06 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993832+0000", 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001620565 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993886+0000", 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4021999999999999e-05 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994477+0000", 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059046499999999998 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995906+0000", 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014291919999999999 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034552+0000", 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038646106999999999 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034737+0000", 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018489900000000001 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:96 3.as0 3:5bc02ad9:::smithi11731613-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.407361604000002, 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.058410493000000001, 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 96 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:10.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992221+0000", 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2899999999999998e-06 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992229+0000", 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3080000000000008e-06 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992236+0000", 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.528e-06 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994995+0000", 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027588700000000001 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995064+0000", 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9758000000000005e-05 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035940+0000", 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040876066000000003 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036021+0000", 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0243999999999999e-05 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050629+0000", 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014608009999999999 2024-09-16T06:49:10.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=18, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.404279856000002, 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040520444000000003, 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.439 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.440 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995299+0000", 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995304+0000", 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5680000000000001e-06 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.441 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034371+0000", 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039067118999999997 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034380+0000", 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7949999999999996e-06 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034448+0000", 2024-09-16T06:49:10.442 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8244000000000006e-05 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035820+0000", 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001372306 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:144 3.as0 3:50fa89c4:::smithi11731613-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.335893386999999, 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38453364499999998, 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 144 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063691+0000", 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8879999999999996e-06 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063699+0000", 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5799999999999992e-06 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063705+0000", 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.246e-06 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.446 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065212+0000", 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015062820000000001 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065312+0000", 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010052600000000001 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.447 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065706+0000", 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039373300000000001 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070934+0000", 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052282070000000003 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448115+0000", 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37718074400000001 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.448 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448220+0000", 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105104 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:145 3.as0 3:50fa89c4:::smithi11731613-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.335850418, 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38866241499999998, 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.449 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 145 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.450 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063732+0000", 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1999999999999999e-06 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.451 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063741+0000", 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4700000000000008e-06 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063748+0000", 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4740000000000002e-06 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066083+0000", 2024-09-16T06:49:10.452 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023351140000000001 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066155+0000", 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1650999999999996e-05 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448700+0000", 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38254526700000002 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.453 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448854+0000", 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154306 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.452392+0000", 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035374640000000001 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.454 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=23, last_complete=35'13, committed=1, applied=1))", 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.328585208, 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37764362699999998, 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.455 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070994+0000", 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.071000+0000", 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8980000000000001e-06 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447985+0000", 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37698559799999998 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.457 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447996+0000", 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0672999999999999e-05 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448044+0000", 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8338999999999999e-05 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448638+0000", 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059397499999999995 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.458 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:302 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 2] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 40.264717060999999, 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21732169600000001, 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 302 2024-09-16T06:49:10.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.460 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134859+0000", 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134878+0000", 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9264999999999999e-05 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134922+0000", 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3260000000000003e-05 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.461 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134944+0000", 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2603e-05 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135181+0000", 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023672400000000001 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135385+0000", 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020434300000000001 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.462 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.347993+0000", 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21260742799999999 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.351735+0000", 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037421830000000001 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352007+0000", 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027182200000000001 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.463 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352184+0000", 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017777400000000001 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:303 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 40.264610890999997, 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21552337099999999, 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.464 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 303 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.465 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134967+0000", 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134976+0000", 2024-09-16T06:49:10.466 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3860000000000007e-06 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134996+0000", 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0234999999999999e-05 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135015+0000", 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8949000000000001e-05 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.467 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135785+0000", 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000770388 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135925+0000", 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013959700000000001 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.468 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350066+0000", 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21414113400000001 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350271+0000", 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020513000000000001 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.469 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350492+0000", 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022106200000000001 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s2 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.836681+0000", 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 33.562898746000002, 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.020203395999999998, 2024-09-16T06:49:10.470 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836681+0000", 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836681+0000", 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.471 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836680+0000", 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836717+0000", 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6936e-05 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836965+0000", 2024-09-16T06:49:10.472 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00024771 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.836975+0000", 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0896999999999999e-05 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.839828+0000", 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002852135 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.473 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.839853+0000", 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5069e-05 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856839+0000", 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016986543 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.856884+0000", 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5213999999999998e-05 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.474 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:368 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (57) in=7b] snapc e=[e,d,c,b,9,8,6,5] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.891488+0000", 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 33.508091903999997, 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025167017999999999, 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 368 2024-09-16T06:49:10.475 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891488+0000", 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891488+0000", 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891486+0000", 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891496+0000", 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0237e-05 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891516+0000", 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0307999999999999e-05 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.477 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.891531+0000", 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4513e-05 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.894517+0000", 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0029859410000000002 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.894851+0000", 2024-09-16T06:49:10.478 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0003346 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.896033+0000", 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001181489 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.898299+0000", 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022659519999999999 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.479 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.916437+0000", 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018137820999999998 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.916655+0000", 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021813400000000001 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.480 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.fs5 61/37 ECSubWrite(tid=159, reqid=osd.11.0:159, at_version=61'68, trim_to=0'0, roll_forward_to=57'64))", 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.828828+0000", 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 21.570751344000001, 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018879105, 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828828+0000", 2024-09-16T06:49:10.481 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828828+0000", 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828826+0000", 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.482 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828884+0000", 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8081000000000002e-05 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.828997+0000", 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112429 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.829020+0000", 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3366e-05 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.483 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.830864+0000", 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0018436489999999999 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.830882+0000", 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8626000000000001e-05 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.484 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.847640+0000", 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016757398 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.847708+0000", 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7853000000000002e-05 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.485 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 61/32 ECSubWriteReply(tid=126, last_complete=61'99, committed=1, applied=1))", 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 21.523897253000001, 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63424799499999995, 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875681+0000", 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.487 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875692+0000", 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1154e-05 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508662+0000", 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63297013099999999 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508675+0000", 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3217000000000001e-05 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.488 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508759+0000", 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3652999999999993e-05 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.509930+0000", 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001171486 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.489 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:542 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.57116856, 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031510007, 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 542 2024-09-16T06:49:10.490 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.491 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828399+0000", 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828425+0000", 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5700000000000001e-05 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828469+0000", 2024-09-16T06:49:10.492 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3510999999999997e-05 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828493+0000", 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4769999999999998e-05 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828815+0000", 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032114900000000001 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.493 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828827+0000", 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2143999999999999e-05 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835549+0000", 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.006721856 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.494 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835824+0000", 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027592399999999998 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.838011+0000", 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021865410000000002 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.849034+0000", 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011023141 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.495 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.496 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859755+0000", 2024-09-16T06:49:10.496 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010721039999999999 2024-09-16T06:49:10.496 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.496 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.496 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.496 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859921+0000", 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016604799999999999 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.497 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:543 3.6s0 3:64026cd9:::smithi11731613-35:head [rollback 24] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.571050324, 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034430898000000001, 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 543 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.498 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828528+0000", 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.499 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828538+0000", 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0457e-05 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828557+0000", 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8459000000000001e-05 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828571+0000", 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.447e-05 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.500 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828929+0000", 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035788200000000001 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828944+0000", 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4662e-05 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840136+0000", 2024-09-16T06:49:10.501 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011191942 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840414+0000", 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027797100000000001 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.842038+0000", 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0016248160000000001 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.502 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855171+0000", 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013132853 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862552+0000", 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073803990000000002 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862960+0000", 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00040862400000000001 2024-09-16T06:49:10.503 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:544 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.570995756, 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034849585000000002, 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.504 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 544 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:10.505 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828583+0000", 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828589+0000", 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5819999999999998e-06 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.506 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828607+0000", 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7260000000000001e-05 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828620+0000", 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3789e-05 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828991+0000", 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037102199999999999 2024-09-16T06:49:10.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828999+0000", 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5000000000000002e-06 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843716+0000", 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0147174 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.508 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843970+0000", 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000253732 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845224+0000", 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001253524 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855177+0000", 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0099530999999999994 2024-09-16T06:49:10.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863271+0000", 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0080939529999999992 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863434+0000", 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016288899999999999 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.510 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.511 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:10.512 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:49:10.513 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:10.512+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:10.613 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:10.616+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:10.640 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:10.641 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:10.641 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:10.641 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:10.654 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:49:10.714 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:10.712+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:10.774 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:10.774 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:10.775 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:10.775 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:10.775 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:10.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:49:10.815 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:10.811+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:10.843 INFO:tasks.rados.rados.0.smithi117.stdout:289: done (4 left) 2024-09-16T06:49:10.843 INFO:tasks.rados.rados.0.smithi117.stdout:290: copy_from oid 30 from oid 45 current snap is 30 2024-09-16T06:49:10.849 INFO:tasks.rados.rados.0.smithi117.stdout:291: snap_create 2024-09-16T06:49:10.849 INFO:tasks.rados.rados.0.smithi117.stdout:284: finishing rollback tid 1 to smithi11731613-3 2024-09-16T06:49:10.849 INFO:tasks.rados.rados.0.smithi117.stdout:285: finishing copy_from racing read to smithi11731613-25 2024-09-16T06:49:10.849 INFO:tasks.rados.rados.0.smithi117.stdout:285: finishing copy_from to smithi11731613-25 2024-09-16T06:49:10.850 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 25 v 97 (ObjNum 121 snap 28 seq_num 121) dirty exists 2024-09-16T06:49:10.850 INFO:tasks.rados.rados.0.smithi117.stdout:284: finishing rollback tid 2 to smithi11731613-3 2024-09-16T06:49:10.850 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 47 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-16T06:49:10.850 INFO:tasks.rados.rados.0.smithi117.stdout:288: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:49:10.854 INFO:tasks.rados.rados.0.smithi117.stdout:287: expect (ObjNum 117 snap 27 seq_num 117) 2024-09-16T06:49:10.855 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:49:10.916 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:10.914+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:1 3.ds0 3:b273fc7d:::smithi11731613-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.025719739000003, 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011215246, 2024-09-16T06:49:10.920 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 1 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.921 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892976+0000", 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.892973+0000", 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893050+0000", 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6996999999999995e-05 2024-09-16T06:49:10.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893075+0000", 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5406999999999999e-05 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893092+0000", 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6541999999999998e-05 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.923 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893183+0000", 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1404999999999996e-05 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893334+0000", 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015069399999999999 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894090+0000", 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00075619499999999998 2024-09-16T06:49:10.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.901851+0000", 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0077611449999999997 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012789380000000001 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.904191+0000", 2024-09-16T06:49:10.925 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001060636 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.025585774, 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.926 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:10.927 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.928 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.929 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:49:10.930 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.025501978000001, 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.932 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.934 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:49:10.936 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.994096739, 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.937 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.938 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.939 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:49:10.940 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.942 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.994046971000003, 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.943 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:49:10.945 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:10.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.886309306000001, 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.948 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.949 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.950 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:49:10.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.953 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.885753846999997, 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:49:10.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:49:10.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:49:10.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.959 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.885697516, 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.960 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:49:10.962 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:10.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:219 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [copy-from ver 4] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.962478365999999, 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031057211000000001, 2024-09-16T06:49:10.965 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 219 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956214+0000", 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956234+0000", 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0275000000000002e-05 2024-09-16T06:49:10.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956284+0000", 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0219999999999997e-05 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956309+0000", 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4930999999999999e-05 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956442+0000", 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013270999999999999 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956459+0000", 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7317999999999999e-05 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960744+0000", 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0042850090000000002 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961009+0000", 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026485300000000001 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.976584+0000", 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015575011999999999 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.981658+0000", 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050738470000000003 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987101+0000", 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054435580000000003 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987274+0000", 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017305700000000001 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:220 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.962355907999999, 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023931746, 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 220 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.972 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956338+0000", 2024-09-16T06:49:10.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956351+0000", 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2849e-05 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956375+0000", 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4195000000000001e-05 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956391+0000", 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5352000000000001e-05 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956650+0000", 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025920099999999997 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956669+0000", 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9157999999999999e-05 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961818+0000", 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051493709999999998 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961948+0000", 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012956 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979661+0000", 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017713421 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979887+0000", 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022550399999999999 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.980271+0000", 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038454100000000002 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.977 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:316 3.ds0 3:b2c197ab:::smithi11731613-17:head [copy-from ver 4] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 40.756680404999997, 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017020408000000001, 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 316 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.978 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162012+0000", 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162030+0000", 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8688000000000001e-05 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162067+0000", 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6526000000000002e-05 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162083+0000", 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6061000000000001e-05 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162190+0000", 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107351 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162511+0000", 2024-09-16T06:49:10.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032124399999999999 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.174275+0000", 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011763691999999999 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.176204+0000", 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019286329999999999 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.178962+0000", 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027583099999999999 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.179036+0000", 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3456999999999998e-05 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.983 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:317 3.ds0 3:b2c197ab:::smithi11731613-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 40.756595070000003, 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013682241, 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 317 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.984 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162106+0000", 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4620000000000001e-06 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162124+0000", 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7706999999999998e-05 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162136+0000", 2024-09-16T06:49:10.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2285e-05 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162962+0000", 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000826123 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163038+0000", 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5371999999999996e-05 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175269+0000", 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01223169 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175524+0000", 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025445300000000003 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175783+0000", 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025890899999999998 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.988 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:353 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 36.076213441, 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011292597, 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.989 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 353 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842482+0000", 2024-09-16T06:49:10.990 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842481+0000", 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842490+0000", 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4760000000000004e-06 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842509+0000", 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8924999999999999e-05 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842523+0000", 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4446e-05 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.843021+0000", 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000497589 2024-09-16T06:49:10.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.843096+0000", 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5332000000000003e-05 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.853775+0000", 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010678309 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:354 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 36.076157512000002, 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013614067000000001, 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:10.994 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 354 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842537+0000", 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842545+0000", 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2500000000000006e-06 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842562+0000", 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7512000000000001e-05 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842573+0000", 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0976e-05 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844555+0000", 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001981919 2024-09-16T06:49:10.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844621+0000", 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5792999999999998e-05 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.856152+0000", 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011530883 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:10.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=96, reqid=osd.10.0:96, at_version=71'80, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.077960561999999, 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012334845000000001, 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:10.999 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840734+0000", 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840800+0000", 2024-09-16T06:49:11.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5324e-05 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840986+0000", 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000186701 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841011+0000", 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4372e-05 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841156+0000", 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000145749 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841215+0000", 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8845000000000003e-05 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853010+0000", 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011794392000000001 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853070+0000", 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0158000000000002e-05 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=97, reqid=osd.10.0:97, at_version=71'82, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.077888051, 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013582399, 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.005 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840851+0000", 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4564999999999998e-05 2024-09-16T06:49:11.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841065+0000", 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000213997 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841082+0000", 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6833e-05 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845856+0000", 2024-09-16T06:49:11.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0047738549999999996 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845879+0000", 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854322+0000", 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084429220000000003 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854390+0000", 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7514000000000004e-05 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 71/32 ECSubWrite(tid=145, reqid=osd.3.0:145, at_version=71'117, trim_to=0'0, roll_forward_to=69'113))", 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.841387+0000", 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.077308336, 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010776747999999999, 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.009 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841387+0000", 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841387+0000", 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841385+0000", 2024-09-16T06:49:11.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841438+0000", 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3096000000000001e-05 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841534+0000", 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.5647000000000001e-05 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.011 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841556+0000", 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2221999999999998e-05 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841678+0000", 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000121373 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841703+0000", 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5602e-05 2024-09-16T06:49:11.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.852102+0000", 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010399212999999999 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.852164+0000", 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.1495000000000007e-05 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.013 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 71/37 ECSubWrite(tid=20, reqid=osd.5.0:20, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.075533362, 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013186145, 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:11.014 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843211+0000", 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9116000000000002e-05 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843322+0000", 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000111142 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843344+0000", 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2350000000000001e-05 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848942+0000", 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005598235 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848970+0000", 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7875999999999999e-05 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856277+0000", 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073063049999999999 2024-09-16T06:49:11.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856348+0000", 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1741000000000003e-05 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 72/32 ECSubWrite(tid=101, reqid=osd.10.0:101, at_version=72'86, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837785+0000", 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.080910626, 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010729579, 2024-09-16T06:49:11.018 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837785+0000", 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837785+0000", 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837784+0000", 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837854+0000", 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9890999999999999e-05 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837967+0000", 2024-09-16T06:49:11.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113093 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837991+0000", 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3272e-05 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838292+0000", 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030179399999999998 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838314+0000", 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1396e-05 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.848444+0000", 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010130402 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.848514+0000", 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0230999999999999e-05 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.022 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds0 72/54 ECSubWrite(tid=156, reqid=osd.3.0:156, at_version=72'61, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.08080445, 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013858534, 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837890+0000", 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837941+0000", 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0958999999999998e-05 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838125+0000", 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000184014 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838144+0000", 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8258999999999999e-05 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838277+0000", 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000133452 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838303+0000", 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6038999999999999e-05 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851693+0000", 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013389803 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.026 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851750+0000", 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6641000000000002e-05 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.027 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:11.029 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:49:11.029 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:11.022+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:11.129 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:11.126+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:11.146 INFO:tasks.rados.rados.0.smithi117.stdout:290: finishing copy_from racing read to smithi11731613-30 2024-09-16T06:49:11.146 INFO:tasks.rados.rados.0.smithi117.stdout:290: finishing copy_from to smithi11731613-30 2024-09-16T06:49:11.146 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 34 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-16T06:49:11.190 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:11.190 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:11.190 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:11.190 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:3.es6 2024-09-16T06:49:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:3.cs2 2024-09-16T06:49:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:3.8s4 2024-09-16T06:49:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:3.5s2 2024-09-16T06:49:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:3.ds6 2024-09-16T06:49:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:2.5 2024-09-16T06:49:11.203 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:49:11.230 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:11.227+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:11.331 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:11.327+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:11.368 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:11.368 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:11.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:49:11.433 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:11.430+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:21 3.fs0 3:fa1c58b4:::smithi11731613-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.61081145, 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.036267527000000001, 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.509 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 21 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.510 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895701+0000", 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895700+0000", 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.511 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895704+0000", 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.048e-06 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895714+0000", 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8509999999999994e-06 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895721+0000", 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2339999999999997e-06 2024-09-16T06:49:11.512 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913213+0000", 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017491642000000002 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913291+0000", 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8053999999999995e-05 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930092+0000", 2024-09-16T06:49:11.513 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016801100999999999 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.930264+0000", 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017232 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.931968+0000", 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017039620000000001 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.514 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:25 3.fs0 3:f3961c35:::smithi11731613-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.610783640999998, 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.036948371000000001, 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.515 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 25 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.516 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895728+0000", 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895853+0000", 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000125451 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895874+0000", 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0346000000000001e-05 2024-09-16T06:49:11.517 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895883+0000", 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0999999999999993e-06 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913349+0000", 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017466612999999999 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.518 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.913577+0000", 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022744 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.914548+0000", 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00097095700000000005 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917700+0000", 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031521740000000002 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.519 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.932545+0000", 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014845191000000001 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.932677+0000", 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013169899999999999 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.520 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:26 3.fs0 3:f3961c35:::smithi11731613-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.610620255000001, 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.037776441000000001, 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 26 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.521 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895892+0000", 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895891+0000", 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:11.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895896+0000", 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9599999999999999e-06 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895906+0000", 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9969999999999993e-06 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895913+0000", 2024-09-16T06:49:11.523 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.08e-06 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915339+0000", 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.019426282 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.915437+0000", 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7822e-05 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.524 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916103+0000", 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00066603099999999998 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917703+0000", 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015995969999999999 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.933600+0000", 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015897047000000001 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.525 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.933668+0000", 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8465000000000003e-05 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:27 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.610591366999998, 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039993954999999998, 2024-09-16T06:49:11.526 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 27 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.527 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895921+0000", 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895920+0000", 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895924+0000", 2024-09-16T06:49:11.528 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.506e-06 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895933+0000", 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.876e-06 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895941+0000", 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8229999999999995e-06 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.529 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916469+0000", 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.020528105000000001 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916546+0000", 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6815000000000002e-05 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.934316+0000", 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017769672 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.530 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.934388+0000", 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1753000000000002e-05 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.935915+0000", 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015271079999999999 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.531 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 35/32 ECSubWrite(tid=35, reqid=client.4511.0:152, at_version=35'10, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 48.428672405999997, 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.44133388099999998, 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.532 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077839+0000", 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077850+0000", 2024-09-16T06:49:11.533 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0505e-05 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515878+0000", 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.43802848300000002 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515883+0000", 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1780000000000002e-06 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.534 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515923+0000", 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9866000000000001e-05 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515930+0000", 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.224e-06 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519096+0000", 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031664449999999999 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.535 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519174+0000", 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7590999999999996e-05 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:163 3.cs0 3:31c9f1c0:::smithi11731613-46:head [copy-from ver 2 in=54b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.994430715999997, 2024-09-16T06:49:11.536 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.80778171600000004, 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 163 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.538 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512085+0000", 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8870000000000002e-06 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512091+0000", 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3539999999999996e-06 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512096+0000", 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8029999999999996e-06 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513808+0000", 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001712199 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513857+0000", 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8649000000000001e-05 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.524232+0000", 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010375422 2024-09-16T06:49:11.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.527189+0000", 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002956939 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319694+0000", 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79250512900000003 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.541 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319863+0000", 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168796 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.979803646999997, 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79400328899999995, 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.542 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526711+0000", 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.18e-06 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319361+0000", 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79264922400000004 2024-09-16T06:49:11.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319384+0000", 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3442e-05 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319508+0000", 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123683 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.545 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.320712+0000", 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012041 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 47.977828703, 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.394952786, 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.547 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528687+0000", 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0810000000000004e-06 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923559+0000", 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39487182199999998 2024-09-16T06:49:11.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923572+0000", 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3406e-05 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923624+0000", 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1989e-05 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.549 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923636+0000", 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1990999999999999e-05 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:233 3.cs0 3:3e2de01d:::smithi11731613-47:head [copy-from ver 22] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956752+0000", 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 43.549759692999999, 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042655024, 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.550 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 233 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956752+0000", 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.551 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956752+0000", 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956749+0000", 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956767+0000", 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8751999999999999e-05 2024-09-16T06:49:11.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956819+0000", 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1112999999999997e-05 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956841+0000", 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2410999999999999e-05 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.553 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956968+0000", 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127071 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956981+0000", 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2731999999999999e-05 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960674+0000", 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036936619999999999 2024-09-16T06:49:11.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960877+0000", 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000202368 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.991730+0000", 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.030852876000000001 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.555 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995444+0000", 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037141629999999999 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.999320+0000", 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0038765150000000001 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.556 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.999407+0000", 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7151000000000006e-05 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:234 3.cs0 3:3e2de01d:::smithi11731613-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956880+0000", 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 43.549632256000002, 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038279512000000002, 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.557 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 234 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956880+0000", 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.558 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956880+0000", 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956878+0000", 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956887+0000", 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3389999999999993e-06 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.559 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956910+0000", 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3587999999999998e-05 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956927+0000", 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6905000000000002e-05 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.957104+0000", 2024-09-16T06:49:11.560 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017687099999999999 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.957114+0000", 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4849999999999998e-06 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961213+0000", 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0040992349999999997 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.561 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961336+0000", 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123201 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.994816+0000", 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033479832000000001 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995000+0000", 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018455000000000001 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.562 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.995159+0000", 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015890199999999999 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=97, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.661712543, 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15568405900000001, 2024-09-16T06:49:11.563 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844798+0000", 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844807+0000", 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8319999999999995e-06 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.998993+0000", 2024-09-16T06:49:11.565 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15418552099999999 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999006+0000", 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3069e-05 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999149+0000", 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000142965 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.566 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000484+0000", 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013350300000000001 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 49/37 ECSubWriteReply(tid=98, last_complete=49'26, committed=1, applied=1))", 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.656001621000001, 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15060995699999999, 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.567 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.568 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850509+0000", 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850523+0000", 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4219000000000001e-05 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000937+0000", 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15041373399999999 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.569 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000960+0000", 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3768e-05 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001073+0000", 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112218 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001120+0000", 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7936999999999999e-05 2024-09-16T06:49:11.570 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as6 49/32 ECSubWrite(tid=80, reqid=client.4511.0:373, at_version=49'61, trim_to=0'0, roll_forward_to=49'59))", 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.574733322, 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.071706169, 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.571 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931778+0000", 2024-09-16T06:49:11.572 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931805+0000", 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7250000000000002e-05 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002139+0000", 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.070333802000000001 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002154+0000", 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5176e-05 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002259+0000", 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105357 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002284+0000", 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4743e-05 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.003425+0000", 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001140605 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.003485+0000", 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0442999999999997e-05 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=99, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.504409604999999, 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123751468, 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.576 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002101+0000", 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002109+0000", 2024-09-16T06:49:11.577 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8399999999999995e-06 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125675+0000", 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123566417 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125711+0000", 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5215000000000001e-05 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125821+0000", 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110661 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125854+0000", 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2638000000000002e-05 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:428 3.cs0 3:31c9f1c0:::smithi11731613-46:head [delete] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:11.579 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 30.651610356999999, 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.62638514099999998, 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 428 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:11.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854899+0000", 2024-09-16T06:49:11.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854915+0000", 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5262999999999999e-05 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854948+0000", 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3005e-05 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.582 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854965+0000", 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7112000000000002e-05 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855070+0000", 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105198 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855081+0000", 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1308000000000001e-05 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478008+0000", 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.622927122 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478196+0000", 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000187534 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.481287+0000", 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030910360000000001 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(osd.5.22:1 3.fs0 3:fa1c58b4:::smithi11731613-7:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 30.637832116999999, 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.61158470300000001, 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:11.585 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868678+0000", 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868690+0000", 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2336e-05 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.587 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868722+0000", 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2122000000000002e-05 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868744+0000", 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1855e-05 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.588 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868876+0000", 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013210300000000001 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868888+0000", 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2053e-05 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478066+0000", 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.60917769200000005 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478207+0000", 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000140743 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.480265+0000", 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020579219999999998 2024-09-16T06:49:11.590 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:431 3.fs0 3:f3961c35:::smithi11731613-9:head [stat,delete] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 28.691531982000001, 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.106237357, 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.591 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 431 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814977+0000", 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814993+0000", 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6095000000000001e-05 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.593 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815025+0000", 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1822000000000001e-05 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815042+0000", 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6803000000000002e-05 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815181+0000", 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013884600000000001 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.594 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815482+0000", 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030067499999999998 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.817743+0000", 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022618769999999998 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.820149+0000", 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024051290000000002 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.595 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921014+0000", 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.10086516700000001 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921218+0000", 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020381300000000001 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.596 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:583 3.fs0 3:fa1c58b4:::smithi11731613-7:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.447180655, 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41724843499999997, 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 583 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.597 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:11.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059336+0000", 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5140000000000001e-06 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059357+0000", 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0602e-05 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.599 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059374+0000", 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7640000000000001e-05 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062783+0000", 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003409081 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062871+0000", 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7188999999999995e-05 2024-09-16T06:49:11.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.063266+0000", 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039578200000000001 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.064116+0000", 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00084932599999999997 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:11.601 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476470+0000", 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41235417099999999 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476580+0000", 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011006999999999999 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 72/37 ECSubWriteReply(tid=209, last_complete=72'94, committed=1, applied=1))", 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:11.602 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.440952749999999, 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41187659199999999, 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:11.603 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065564+0000", 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2839999999999997e-06 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.604 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476221+0000", 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41065675400000001 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476236+0000", 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5509e-05 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476326+0000", 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9337000000000002e-05 2024-09-16T06:49:11.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.477436+0000", 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011104210000000001 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.1s1 81/77 ECSubWrite(tid=85, reqid=client.4511.0:732, at_version=0'0, trim_to=0'0, roll_forward_to=81'47))", 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 2.1998886789999998, 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38681583800000002, 2024-09-16T06:49:11.606 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.607 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306621+0000", 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306649+0000", 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7733e-05 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692262+0000", 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.385613345 2024-09-16T06:49:11.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692278+0000", 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5642e-05 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692421+0000", 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014278699999999999 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692441+0000", 2024-09-16T06:49:11.609 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0089e-05 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693380+0000", 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00093900299999999999 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693439+0000", 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9342000000000002e-05 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.610 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:11.611 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:11.611 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:11.612 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:49:11.613 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:11.611+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:11.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:11.713 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:11.710+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:11.739 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.8s4 --file /var/log/ceph/exp.3.8s4.5 2024-09-16T06:49:11.780 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:11.780 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:11.780 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:11.780 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:11.794 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:49:11.810 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:11.811 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:11.814 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:11.816+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:11.832 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:11.832 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:11.854 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:49:11.859 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:11.859 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:11.860 INFO:tasks.rados.rados.0.smithi117.stdout:284: done (5 left) 2024-09-16T06:49:11.860 INFO:tasks.rados.rados.0.smithi117.stdout:285: done (4 left) 2024-09-16T06:49:11.860 INFO:tasks.rados.rados.0.smithi117.stdout:287: done (3 left) 2024-09-16T06:49:11.860 INFO:tasks.rados.rados.0.smithi117.stdout:288: done (2 left) 2024-09-16T06:49:11.860 INFO:tasks.rados.rados.0.smithi117.stdout:290: done (1 left) 2024-09-16T06:49:11.860 INFO:tasks.rados.rados.0.smithi117.stdout:291: done (0 left) 2024-09-16T06:49:11.861 INFO:tasks.rados.rados.0.smithi117.stdout:292: snap_remove snap 20 2024-09-16T06:49:11.883 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:11.883 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:11.903 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:11.903 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:11.903 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:11.915 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:11.910+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:11.928 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:11.955 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:11.955 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:11.962 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:11.963 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:11.963 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:11.963 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:11.963 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:11.978 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:49:11.981 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:11.981 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:12.010 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:12.010 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:12.016 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:12.014+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:12.041 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:12.041 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:12.069 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:12.069 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:12.098 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:12.098 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:12.116 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:12.115+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:12.129 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:12.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:12.132+0000 7fb972ffd640 0 -- 172.21.15.100:0/311484730 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7fb97421eb10 msgr2=0x7fb974220f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:266 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 43.188411914, 2024-09-16T06:49:12.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014774862999999999, 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 266 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:12.146 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953825+0000", 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953846+0000", 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0171999999999999e-05 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953889+0000", 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3284000000000001e-05 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953913+0000", 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4071e-05 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954055+0000", 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014240699999999999 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.149 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954076+0000", 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0678000000000001e-05 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961148+0000", 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070716959999999997 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961402+0000", 2024-09-16T06:49:12.150 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025470100000000003 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.968604+0000", 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072013340000000002 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:267 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 43.188295701999998, 2024-09-16T06:49:12.151 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015485804000000001, 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 267 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.152 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.153 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.153 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:12.153 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.154 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:12.154 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953944+0000", 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953973+0000", 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9581e-05 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953995+0000", 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1685e-05 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954011+0000", 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5862000000000002e-05 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954223+0000", 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021187199999999999 2024-09-16T06:49:12.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954233+0000", 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0789e-05 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962540+0000", 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0083068819999999998 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962710+0000", 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017007799999999999 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.969431+0000", 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0067204250000000004 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.158 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:268 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 43.188215454999998, 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.30266284300000001, 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 268 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.159 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954024+0000", 2024-09-16T06:49:12.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954032+0000", 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4290000000000004e-06 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954050+0000", 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8332e-05 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954063+0000", 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2955999999999999e-05 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954379+0000", 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031616000000000001 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954389+0000", 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3959999999999995e-06 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963428+0000", 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0090396959999999998 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963723+0000", 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00029431800000000001 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:29.256688+0000", 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.29296564200000003 2024-09-16T06:49:12.163 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.306953266000001, 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014247134, 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.164 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.165 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835286+0000", 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835349+0000", 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3258999999999999e-05 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835459+0000", 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110097 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835480+0000", 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1424999999999999e-05 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835600+0000", 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011922 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835623+0000", 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3144999999999999e-05 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849454+0000", 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013830779 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849535+0000", 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1021999999999998e-05 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.168 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 61/32 ECSubWrite(tid=83, reqid=osd.14.0:83, at_version=61'72, trim_to=0'0, roll_forward_to=59'68))", 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 23.308668089000001, 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012777472999999999, 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:12.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833571+0000", 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.170 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833631+0000", 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9917999999999998e-05 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833761+0000", 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013055799999999999 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833784+0000", 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2714999999999999e-05 2024-09-16T06:49:12.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835926+0000", 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021422720000000002 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835943+0000", 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6889e-05 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846281+0000", 2024-09-16T06:49:12.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010337641999999999 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846350+0000", 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9315999999999998e-05 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 61/32 ECSubWrite(tid=85, reqid=osd.10.0:85, at_version=61'70, trim_to=0'0, roll_forward_to=59'66))", 2024-09-16T06:49:12.173 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 23.303786356, 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013138069, 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838452+0000", 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838506+0000", 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3594999999999998e-05 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839270+0000", 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00076359400000000003 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839293+0000", 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3006e-05 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839420+0000", 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127355 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839446+0000", 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851525+0000", 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012079196 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851592+0000", 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7156999999999994e-05 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=101, reqid=osd.10.0:101, at_version=72'86, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:12.178 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.304773162, 2024-09-16T06:49:12.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017459348, 2024-09-16T06:49:12.179 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.179 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.179 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:12.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837467+0000", 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837538+0000", 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0541000000000001e-05 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837664+0000", 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012650899999999999 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.182 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837687+0000", 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2759999999999999e-05 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839677+0000", 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019901300000000001 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839707+0000", 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0049e-05 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854862+0000", 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015155366 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854927+0000", 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4629999999999996e-05 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.184 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=157, reqid=osd.3.0:157, at_version=72'123, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.304460209, 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027234286, 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837780+0000", 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837839+0000", 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8801000000000002e-05 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837959+0000", 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000120436 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837982+0000", 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3156e-05 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838118+0000", 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000135673 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838143+0000", 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4899e-05 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.864948+0000", 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026805539999999999 2024-09-16T06:49:12.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865015+0000", 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6501000000000006e-05 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=102, reqid=osd.10.0:102, at_version=72'88, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.301649225, 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018351104, 2024-09-16T06:49:12.189 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:12.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840590+0000", 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840640+0000", 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0631999999999999e-05 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840728+0000", 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7933e-05 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840750+0000", 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2022999999999999e-05 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840864+0000", 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113423 2024-09-16T06:49:12.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840889+0000", 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5092999999999998e-05 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858883+0000", 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017994231999999999 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858943+0000", 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9587999999999999e-05 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=158, reqid=osd.3.0:158, at_version=72'125, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.300008088, 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022836532999999999, 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.194 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842231+0000", 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:12.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842295+0000", 2024-09-16T06:49:12.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4238000000000004e-05 2024-09-16T06:49:12.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842418+0000", 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012244299999999999 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842441+0000", 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3606000000000001e-05 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850726+0000", 2024-09-16T06:49:12.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082850219999999995 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850750+0000", 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3822e-05 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865034+0000", 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014283563000000001 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865069+0000", 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5712e-05 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=65, reqid=osd.12.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.296685063, 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017333787, 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.200 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845555+0000", 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845609+0000", 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4284999999999999e-05 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845766+0000", 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015658100000000001 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845789+0000", 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2866000000000001e-05 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845942+0000", 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015326200000000001 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845969+0000", 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7296000000000001e-05 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862827+0000", 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016857351999999999 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862890+0000", 2024-09-16T06:49:12.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2997999999999996e-05 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=66, reqid=osd.12.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.294993577, 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017965090999999999, 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847246+0000", 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847298+0000", 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2677999999999999e-05 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847412+0000", 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113403 2024-09-16T06:49:12.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847437+0000", 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4973e-05 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857022+0000", 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0095850889999999998 2024-09-16T06:49:12.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857045+0000", 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3796000000000001e-05 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865168+0000", 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0081221260000000003 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.209 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865212+0000", 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4691999999999998e-05 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=108, reqid=osd.14.0:108, at_version=72'100, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.291894565, 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015047932999999999, 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850339+0000", 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999928 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850413+0000", 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3508000000000002e-05 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850503+0000", 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0303999999999995e-05 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850526+0000", 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2595999999999998e-05 2024-09-16T06:49:12.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850694+0000", 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168367 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850718+0000", 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.44e-05 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865344+0000", 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014625658999999999 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865394+0000", 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0074999999999997e-05 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=109, reqid=osd.14.0:109, at_version=72'102, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.288246997, 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013551083, 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853992+0000", 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854050+0000", 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8495000000000002e-05 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854252+0000", 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020159999999999999 2024-09-16T06:49:12.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854296+0000", 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4438999999999999e-05 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862592+0000", 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082959690000000003 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862662+0000", 2024-09-16T06:49:12.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9210999999999999e-05 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867494+0000", 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048324029999999999 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867545+0000", 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0838999999999999e-05 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:559 3.5s0 3:af79444c:::smithi11731613-29:1d [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 12.187260839, 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014695526, 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 559 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954979+0000", 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:12.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954984+0000", 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5310000000000003e-06 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954994+0000", 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7249999999999992e-06 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.955000+0000", 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7569999999999998e-06 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967068+0000", 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012068096 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967239+0000", 2024-09-16T06:49:12.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017012400000000001 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.969675+0000", 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024368660000000002 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=191, reqid=osd.6.0:191, at_version=73'31, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848719+0000", 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.293521567999999, 2024-09-16T06:49:12.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01207479, 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848719+0000", 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848719+0000", 2024-09-16T06:49:12.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848718+0000", 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848779+0000", 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0133999999999997e-05 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848977+0000", 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00019813099999999999 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849037+0000", 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9808000000000001e-05 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849244+0000", 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020772999999999999 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849271+0000", 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6652000000000002e-05 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.860732+0000", 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011461176999999999 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.860794+0000", 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.1897999999999996e-05 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=192, reqid=osd.6.0:192, at_version=73'33, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.293454754000001, 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017679872999999999, 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848785+0000", 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848820+0000", 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4931999999999997e-05 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849087+0000", 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026683499999999999 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849103+0000", 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6813e-05 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855706+0000", 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066022570000000003 2024-09-16T06:49:12.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855730+0000", 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4661999999999999e-05 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866426+0000", 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010695893999999999 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866466+0000", 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9632999999999998e-05 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.291328851999999, 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015506908999999999, 2024-09-16T06:49:12.234 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850911+0000", 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850952+0000", 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0722000000000001e-05 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851117+0000", 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000164599 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851132+0000", 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5255e-05 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854555+0000", 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034234320000000001 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854579+0000", 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866327+0000", 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011748364000000001 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866419+0000", 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1624000000000002e-05 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.238 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 73/32 ECSubWrite(tid=107, reqid=osd.10.0:107, at_version=73'90, trim_to=0'0, roll_forward_to=72'88))", 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.288076381, 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012199229000000001, 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:12.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:12.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854217+0000", 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3168000000000003e-05 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854327+0000", 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109833 2024-09-16T06:49:12.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854349+0000", 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2296000000000001e-05 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855253+0000", 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090369500000000004 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855283+0000", 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9884999999999999e-05 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866242+0000", 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01095932 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866364+0000", 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000121542 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:705 3.5s0 3:af79444c:::smithi11731613-29:head [append 0~524288] snapc 2b=[2b,2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:12.244 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 6.1085048070000001, 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012998625, 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 705 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:12.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033733+0000", 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:12.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034535+0000", 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00080219500000000001 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034573+0000", 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8473000000000002e-05 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034605+0000", 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1776999999999997e-05 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034755+0000", 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000149641 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.035008+0000", 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025373600000000001 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.039658+0000", 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0046500159999999999 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046264+0000", 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066053680000000004 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046596+0000", 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000331983 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046735+0000", 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013881599999999999 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.250 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.251 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.251 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:12.251 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:12.251 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:12.252 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:49:12.253 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:12.256+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:12.353 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:12.350+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:12.360 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:12.360 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:12.383 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:12.383 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:12.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:49:12.399 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:12.400 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:12.404 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:12.404 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:12.454 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:12.456+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:12.518 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:12.518 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:12.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:49:12.555 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:12.550+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:12.656 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:12.655+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:12.693 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:12.693 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:12.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:49:12.757 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:12.754+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:12.824 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:12.879 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:12.881 INFO:tasks.rados.rados.0.smithi117.stdout:292: done (0 left) 2024-09-16T06:49:12.881 INFO:tasks.rados.rados.0.smithi117.stdout:293: read oid 11 snap -1 2024-09-16T06:49:12.881 INFO:tasks.rados.rados.0.smithi117.stdout:293: expect (ObjNum 121 snap 28 seq_num 121) 2024-09-16T06:49:12.881 INFO:tasks.rados.rados.0.smithi117.stdout:294: read oid 36 snap -1 2024-09-16T06:49:12.881 INFO:tasks.rados.rados.0.smithi117.stdout:294: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:49:12.882 INFO:tasks.rados.rados.0.smithi117.stdout:295: setattr oid 6 current snap is 31 2024-09-16T06:49:12.882 INFO:tasks.rados.rados.0.smithi117.stdout:296: read oid 33 snap -1 2024-09-16T06:49:12.882 INFO:tasks.rados.rados.0.smithi117.stdout:296: expect (ObjNum 74 snap 10 seq_num 74) 2024-09-16T06:49:12.882 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 44 current snap is 31 2024-09-16T06:49:12.882 INFO:tasks.rados.rados.0.smithi117.stdout:297: seq_num 126 ranges {3686400=573440,4259840=622592,4882432=622592,5505024=311296} 2024-09-16T06:49:12.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:49:12.883 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:12.884+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:12.929 INFO:teuthology.orchestra.run.smithi107.stderr:Exporting 3.8s4 info 3.8s4( v 82'38 (0'0,82'38] local-lis/les=32/33 n=4 ec=32/32 lis/c=32/32 les/c/f=33/33/0 sis=32) 2024-09-16T06:49:12.933 INFO:teuthology.orchestra.run.smithi107.stderr:Read 4#3:119b2380:::smithi11731613-26:15# 2024-09-16T06:49:12.935 INFO:teuthology.orchestra.run.smithi107.stderr:Read 4#3:119b2380:::smithi11731613-26:head# 2024-09-16T06:49:12.935 INFO:teuthology.orchestra.run.smithi107.stderr:Read 4#3:1cef29f0:::smithi11731613-40:24# 2024-09-16T06:49:12.936 INFO:teuthology.orchestra.run.smithi107.stderr:Read 4#3:1cef29f0:::smithi11731613-40:head# 2024-09-16T06:49:12.937 INFO:teuthology.orchestra.run.smithi107.stderr:Export successful 2024-09-16T06:49:12.941 INFO:tasks.rados.rados.0.smithi117.stdout:297: writing smithi11731613-44 from 3686400 to 4259840 tid 1 2024-09-16T06:49:12.947 INFO:tasks.rados.rados.0.smithi117.stdout:297: writing smithi11731613-44 from 4259840 to 4882432 tid 2 2024-09-16T06:49:12.953 INFO:tasks.rados.rados.0.smithi117.stdout:297: writing smithi11731613-44 from 4882432 to 5505024 tid 3 2024-09-16T06:49:12.956 INFO:tasks.rados.rados.0.smithi117.stdout:297: writing smithi11731613-44 from 5505024 to 5816320 tid 4 2024-09-16T06:49:12.956 INFO:tasks.rados.rados.0.smithi117.stdout:298: snap_create 2024-09-16T06:49:12.956 INFO:tasks.rados.rados.0.smithi117.stdout:294: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:49:12.982 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:12.978+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:12.993 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 6 v 19 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:12.993 INFO:tasks.rados.rados.0.smithi117.stdout:293: expect (ObjNum 121 snap 28 seq_num 121) 2024-09-16T06:49:13.052 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.052 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.064 INFO:tasks.rados.rados.0.smithi117.stdout:296: expect (ObjNum 74 snap 10 seq_num 74) 2024-09-16T06:49:13.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:49:13.082 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:13.078+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:13.111 INFO:tasks.rados.rados.0.smithi117.stdout:297: finishing write tid 1 to smithi11731613-44 2024-09-16T06:49:13.111 INFO:tasks.rados.rados.0.smithi117.stdout:297: finishing write tid 2 to smithi11731613-44 2024-09-16T06:49:13.111 INFO:tasks.rados.rados.0.smithi117.stdout:297: finishing write tid 3 to smithi11731613-44 2024-09-16T06:49:13.111 INFO:tasks.rados.rados.0.smithi117.stdout:297: finishing write tid 4 to smithi11731613-44 2024-09-16T06:49:13.111 INFO:tasks.rados.rados.0.smithi117.stdout:297: finishing write tid 5 to smithi11731613-44 2024-09-16T06:49:13.112 INFO:tasks.rados.rados.0.smithi117.stdout:297: finishing write tid 6 to smithi11731613-44 2024-09-16T06:49:13.112 INFO:tasks.rados.rados.0.smithi117.stdout:297: oid 44 updating version 0 to 112 2024-09-16T06:49:13.112 INFO:tasks.rados.rados.0.smithi117.stdout:297: oid 44 updating version 112 to 113 2024-09-16T06:49:13.112 INFO:tasks.rados.rados.0.smithi117.stdout:297: oid 44 updating version 113 to 116 2024-09-16T06:49:13.112 INFO:tasks.rados.rados.0.smithi117.stdout:297: oid 44 version 116 is already newer than 115 2024-09-16T06:49:13.113 INFO:tasks.rados.rados.0.smithi117.stdout:297: oid 44 version 116 is already newer than 114 2024-09-16T06:49:13.113 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 116 (ObjNum 126 snap 31 seq_num 126) dirty exists 2024-09-16T06:49:13.113 INFO:tasks.rados.rados.0.smithi117.stdout:297: left oid 44 (ObjNum 126 snap 31 seq_num 126) 2024-09-16T06:49:13.183 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.183 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.183 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:13.182+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:13.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:49:13.284 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:13.283+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:13.318 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.318 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:49:13.385 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:13.383+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:13.449 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.450 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:49:13.478 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.8s4 2024-09-16T06:49:13.507 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:13.512+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:13.580 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.581 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.586 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:13.588+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:13.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:49:13.687 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:13.683+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:13.756 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.756 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.771 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:49:13.787 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:13.783+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:13.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:293: done (5 left) 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:294: done (4 left) 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:295: done (3 left) 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:296: done (2 left) 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:297: done (1 left) 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:298: done (0 left) 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 49 current snap is 32 2024-09-16T06:49:13.880 INFO:tasks.rados.rados.0.smithi117.stdout:299: seq_num 127 ranges {4194304=589824,4784128=753664,5537792=475136,6012928=327680} 2024-09-16T06:49:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:13.894 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:13.896+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:13.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:49:13.917 INFO:tasks.rados.rados.0.smithi117.stdout:299: writing smithi11731613-49 from 4194304 to 4784128 tid 1 2024-09-16T06:49:13.923 INFO:tasks.rados.rados.0.smithi117.stdout:299: writing smithi11731613-49 from 4784128 to 5537792 tid 2 2024-09-16T06:49:13.927 INFO:tasks.rados.rados.0.smithi117.stdout:299: writing smithi11731613-49 from 5537792 to 6012928 tid 3 2024-09-16T06:49:13.930 INFO:tasks.rados.rados.0.smithi117.stdout:299: writing smithi11731613-49 from 6012928 to 6340608 tid 4 2024-09-16T06:49:13.930 INFO:tasks.rados.rados.0.smithi117.stdout:300: copy_from oid 25 from oid 33 current snap is 32 2024-09-16T06:49:13.930 INFO:tasks.rados.rados.0.smithi117.stdout:301: setattr oid 18 current snap is 32 2024-09-16T06:49:13.933 INFO:tasks.rados.rados.0.smithi117.stdout:302: rollback oid 39 current snap is 32 2024-09-16T06:49:13.933 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 39 to 18 2024-09-16T06:49:13.934 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 21 current snap is 32 2024-09-16T06:49:13.934 INFO:tasks.rados.rados.0.smithi117.stdout:303: seq_num 128 ranges {0=606208,606208=507904,1114112=573440,1687552=475136} 2024-09-16T06:49:13.938 INFO:tasks.rados.rados.0.smithi117.stdout:303: writing smithi11731613-21 from 0 to 606208 tid 1 2024-09-16T06:49:13.943 INFO:tasks.rados.rados.0.smithi117.stdout:303: writing smithi11731613-21 from 606208 to 1114112 tid 2 2024-09-16T06:49:13.947 INFO:tasks.rados.rados.0.smithi117.stdout:303: writing smithi11731613-21 from 1114112 to 1687552 tid 3 2024-09-16T06:49:13.951 INFO:tasks.rados.rados.0.smithi117.stdout:303: writing smithi11731613-21 from 1687552 to 2162688 tid 4 2024-09-16T06:49:13.951 INFO:tasks.rados.rados.0.smithi117.stdout:304: snap_remove snap 25 2024-09-16T06:49:13.989 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:13.987+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:14.089 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:14.089 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:14.090 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:14.086+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:14.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:49:14.190 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:14.186+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:14.231 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:14.231 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:14.245 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:49:14.291 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:14.286+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:14.360 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:14.360 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:14.374 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:49:14.392 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:14.396+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:14.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:14.489 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:14.489 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:14.493 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:14.496+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:14.504 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:49:14.560 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:14.560 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:14.584 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:14.584 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:14.594 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:14.591+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:14.608 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:14.625 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:14.625 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:14.634 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:14.634 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:14.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:49:14.660 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:14.692 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:14.692 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:14.694 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:14.696+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:14.696 INFO:teuthology.orchestra.run.smithi107.stdout: marking collection for removal 2024-09-16T06:49:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:setting '_remove' omap key 2024-09-16T06:49:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:finish_remove_pgs 3.8s4_head removing 3.8s4 2024-09-16T06:49:14.701 INFO:teuthology.orchestra.run.smithi107.stdout:Remove successful 2024-09-16T06:49:14.721 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:14.721 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:14.749 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:14.749 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:14.757 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:14.757 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:14.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:49:14.778 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:14.778 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:14.795 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:14.790+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:14.810 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:14.884 INFO:tasks.rados.rados.0.smithi117.stdout:304: done (5 left) 2024-09-16T06:49:14.884 INFO:tasks.rados.rados.0.smithi117.stdout:305: copy_from oid 9 from oid 32 current snap is 32 2024-09-16T06:49:14.884 INFO:tasks.rados.rados.0.smithi117.stdout:299: finishing write tid 1 to smithi11731613-49 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:299: finishing write tid 2 to smithi11731613-49 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 18 v 32 (ObjNum 115 snap 26 seq_num 115) dirty exists 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:299: finishing write tid 3 to smithi11731613-49 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:299: finishing write tid 4 to smithi11731613-49 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:302: finishing rollback tid 0 to smithi11731613-39 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:299: finishing write tid 5 to smithi11731613-49 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:302: finishing rollback tid 1 to smithi11731613-39 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:303: finishing write tid 1 to smithi11731613-21 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:302: finishing rollback tid 2 to smithi11731613-39 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 39 v 141 (ObjNum 70 snap 6 seq_num 70) dirty exists 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:300: finishing copy_from racing read to smithi11731613-25 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:303: finishing write tid 2 to smithi11731613-21 2024-09-16T06:49:14.885 INFO:tasks.rados.rados.0.smithi117.stdout:303: finishing write tid 3 to smithi11731613-21 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:303: finishing write tid 4 to smithi11731613-21 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: finishing write tid 6 to smithi11731613-49 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: oid 49 updating version 0 to 115 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: oid 49 updating version 115 to 116 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: oid 49 updating version 116 to 118 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: oid 49 version 118 is already newer than 114 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: oid 49 version 118 is already newer than 117 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 49 v 118 (ObjNum 127 snap 32 seq_num 127) dirty exists 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:299: left oid 49 (ObjNum 127 snap 32 seq_num 127) 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:303: finishing write tid 5 to smithi11731613-21 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:300: finishing copy_from to smithi11731613-25 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 25 v 47 (ObjNum 74 snap 10 seq_num 74) dirty exists 2024-09-16T06:49:14.886 INFO:tasks.rados.rados.0.smithi117.stdout:303: finishing write tid 6 to smithi11731613-21 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: oid 21 updating version 0 to 116 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: oid 21 updating version 116 to 118 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: oid 21 updating version 118 to 119 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: oid 21 version 119 is already newer than 117 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: oid 21 updating version 119 to 120 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 21 v 120 (ObjNum 128 snap 32 seq_num 128) dirty exists 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: left oid 21 (ObjNum 128 snap 32 seq_num 128) 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:299: done (5 left) 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:300: done (4 left) 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:301: done (3 left) 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:302: done (2 left) 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:303: done (1 left) 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:306: read oid 29 snap -1 2024-09-16T06:49:14.887 INFO:tasks.rados.rados.0.smithi117.stdout:306: expect (ObjNum 123 snap 28 seq_num 123) 2024-09-16T06:49:14.888 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 23 current snap is 32 2024-09-16T06:49:14.888 INFO:tasks.rados.rados.0.smithi117.stdout:307: seq_num 129 ranges {3407872=622592,4030464=655360,4685824=688128,5373952=212992} 2024-09-16T06:49:14.888 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:49:14.894 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 51.994442554999999, 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064369220000000003, 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899647+0000", 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899689+0000", 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2429000000000003e-05 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899768+0000", 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9277999999999995e-05 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899797+0000", 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8473999999999999e-05 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903489+0000", 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036918680000000001 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903519+0000", 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0283000000000001e-05 2024-09-16T06:49:14.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906024+0000", 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025045430000000001 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906085+0000", 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1637000000000006e-05 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 40/37 ECSubWrite(tid=59, reqid=client.4511.0:233, at_version=40'12, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.899768119000001, 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061889309999999999, 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994323+0000", 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.994322+0000", 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995376+0000", 2024-09-16T06:49:14.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00105338 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995464+0000", 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8170000000000005e-05 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995483+0000", 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9548999999999999e-05 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995633+0000", 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014933200000000001 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.995662+0000", 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8847999999999999e-05 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000463+0000", 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048011360000000001 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:28.000512+0000", 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8860999999999999e-05 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 47/37 ECSubWrite(tid=91, reqid=client.4511.0:355, at_version=47'22, trim_to=0'0, roll_forward_to=44'20))", 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 40.047447458999997, 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061968889999999997, 2024-09-16T06:49:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846643+0000", 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846717+0000", 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4350999999999993e-05 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846862+0000", 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014473599999999999 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.846886+0000", 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4289000000000001e-05 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.847145+0000", 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000259158 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.847176+0000", 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1154e-05 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.852788+0000", 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056113070000000003 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:34.852840+0000", 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2509999999999997e-05 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 61/32 ECSubWrite(tid=57, reqid=osd.12.0:57, at_version=61'40, trim_to=0'0, roll_forward_to=59'36))", 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.071230150000002, 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068954469999999999, 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822859+0000", 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822915+0000", 2024-09-16T06:49:14.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6230999999999997e-05 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823009+0000", 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4176000000000001e-05 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823032+0000", 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3005000000000001e-05 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823531+0000", 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000498326 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823611+0000", 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0495999999999996e-05 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829687+0000", 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060763509999999998 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829756+0000", 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8765999999999999e-05 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 16.060955013000001, 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069218320000000002, 2024-09-16T06:49:14.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833135+0000", 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833193+0000", 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8183999999999997e-05 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833314+0000", 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012045900000000001 2024-09-16T06:49:14.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833341+0000", 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6954999999999999e-05 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833966+0000", 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00062526400000000005 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833994+0000", 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7942000000000001e-05 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.840004+0000", 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060096969999999996 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.840058+0000", 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3931000000000003e-05 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.917 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 71/54 ECSubWrite(tid=154, reqid=osd.3.0:149, at_version=0'0, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 16.018500146000001, 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076751120000000004, 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:49:14.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875589+0000", 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875617+0000", 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8209999999999999e-05 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875704+0000", 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7047999999999997e-05 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875725+0000", 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0837000000000001e-05 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875784+0000", 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9077999999999999e-05 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875794+0000", 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5319999999999995e-06 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883218+0000", 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074248020000000003 2024-09-16T06:49:14.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883266+0000", 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7339000000000002e-05 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 71/37 ECSubWrite(tid=129, reqid=osd.1.0:129, at_version=71'79, trim_to=0'0, roll_forward_to=71'77))", 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 16.017984515999999, 2024-09-16T06:49:14.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071128010000000002, 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876105+0000", 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876130+0000", 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.497e-05 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876185+0000", 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4926999999999998e-05 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876194+0000", 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7669999999999992e-06 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876307+0000", 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113189 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876326+0000", 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8553999999999999e-05 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883200+0000", 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006874092 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883219+0000", 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9286000000000001e-05 2024-09-16T06:49:14.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 72/37 ECSubWrite(tid=214, reqid=client.4511.0:614, at_version=72'94, trim_to=0'0, roll_forward_to=72'92))", 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.657014+0000", 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.237077048, 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0097847630000000001, 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657014+0000", 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657014+0000", 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657012+0000", 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657200+0000", 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018793999999999999 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657310+0000", 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011035900000000001 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657333+0000", 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2639000000000001e-05 2024-09-16T06:49:14.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657450+0000", 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116953 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657466+0000", 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5875999999999998e-05 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666720+0000", 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.009254623 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666799+0000", 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.805e-05 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 72/37 ECSubWrite(tid=215, reqid=client.4511.0:615, at_version=72'95, trim_to=0'0, roll_forward_to=72'92))", 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.657860+0000", 2024-09-16T06:49:14.931 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.236230345999999, 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089789369999999993, 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657860+0000", 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657860+0000", 2024-09-16T06:49:14.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657859+0000", 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657921+0000", 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1810999999999995e-05 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658008+0000", 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7837000000000006e-05 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658029+0000", 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658488+0000", 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000459087 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658517+0000", 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9088999999999999e-05 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666817+0000", 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0082997209999999995 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666839+0000", 2024-09-16T06:49:14.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2643e-05 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.9s5 73/32 ECSubWrite(tid=79, reqid=osd.12.0:79, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.042677533000001, 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0095406529999999996, 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851471+0000", 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7877999999999997e-05 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853188+0000", 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017171039999999999 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853213+0000", 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5352000000000001e-05 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858138+0000", 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049245390000000003 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858175+0000", 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7518999999999998e-05 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.860884+0000", 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002708907 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.860954+0000", 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9877999999999997e-05 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.940 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 73/37 ECSubWrite(tid=219, reqid=osd.11.0:219, at_version=73'97, trim_to=0'0, roll_forward_to=72'95))", 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.851564+0000", 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.042526882000001, 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0097268860000000006, 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851564+0000", 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851564+0000", 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851563+0000", 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851666+0000", 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103199 2024-09-16T06:49:14.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851771+0000", 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000104616 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851794+0000", 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2923000000000001e-05 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851983+0000", 2024-09-16T06:49:14.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018950999999999999 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.852011+0000", 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7705e-05 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.861252+0000", 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092412039999999994 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.861291+0000", 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8439000000000002e-05 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.853271+0000", 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.040820008000001, 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010549976000000001, 2024-09-16T06:49:14.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853271+0000", 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853271+0000", 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853269+0000", 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853322+0000", 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3031000000000001e-05 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853536+0000", 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021384900000000001 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853561+0000", 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4998999999999999e-05 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858933+0000", 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053721029999999996 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858973+0000", 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9894999999999997e-05 2024-09-16T06:49:14.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.863755+0000", 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004782204 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.863821+0000", 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5314999999999994e-05 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 73/37 ECSubWrite(tid=152, reqid=osd.1.0:152, at_version=73'89, trim_to=0'0, roll_forward_to=72'87))", 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.853363+0000", 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.040728051, 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0091925289999999996, 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853363+0000", 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853363+0000", 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853362+0000", 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853407+0000", 2024-09-16T06:49:14.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4335000000000002e-05 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853621+0000", 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000214389 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853638+0000", 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7173e-05 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858504+0000", 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048660470000000001 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858533+0000", 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9048999999999999e-05 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862470+0000", 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039370230000000004 2024-09-16T06:49:14.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862555+0000", 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4937000000000003e-05 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/54 ECSubWrite(tid=182, reqid=osd.3.0:182, at_version=73'67, trim_to=0'0, roll_forward_to=72'65))", 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.039849727, 2024-09-16T06:49:14.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0083643499999999996, 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854296+0000", 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5146999999999999e-05 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854438+0000", 2024-09-16T06:49:14.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000142115 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854460+0000", 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2453e-05 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.859010+0000", 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004550119 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.859040+0000", 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9606000000000001e-05 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862528+0000", 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034876859999999998 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862605+0000", 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7733999999999998e-05 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 82/77 ECSubWrite(tid=222, reqid=osd.3.0:219, at_version=0'0, trim_to=0'0, roll_forward_to=82'77))", 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.865487+0000", 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.0286033540000004, 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0082477100000000001, 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865487+0000", 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865487+0000", 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865485+0000", 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865515+0000", 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0229000000000001e-05 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865772+0000", 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025714599999999998 2024-09-16T06:49:14.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865796+0000", 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3938999999999999e-05 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.866003+0000", 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020710100000000001 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.866030+0000", 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6216000000000001e-05 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873679+0000", 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076493029999999997 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873735+0000", 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6276000000000001e-05 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.963 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 82/77 ECSubWrite(tid=186, reqid=osd.1.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=82'114))", 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.867995+0000", 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.0260959090000004, 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090620000000000006, 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.867995+0000", 2024-09-16T06:49:14.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.867995+0000", 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.867993+0000", 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868021+0000", 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8405999999999999e-05 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868127+0000", 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010522 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868149+0000", 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2543000000000001e-05 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868501+0000", 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035184000000000002 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868558+0000", 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6872e-05 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877001+0000", 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084430970000000001 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877057+0000", 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5886e-05 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 82/77 ECSubWrite(tid=187, reqid=osd.1.0:187, at_version=82'116, trim_to=0'0, roll_forward_to=82'114))", 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.869201+0000", 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.024890074, 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079066459999999998, 2024-09-16T06:49:14.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869201+0000", 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869201+0000", 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869199+0000", 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869252+0000", 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3041999999999998e-05 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869351+0000", 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9164000000000001e-05 2024-09-16T06:49:14.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869372+0000", 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1223e-05 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870145+0000", 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077266399999999995 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870174+0000", 2024-09-16T06:49:14.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9218999999999999e-05 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877074+0000", 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068994089999999996 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877107+0000", 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3809000000000001e-05 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 82/81 ECSubWrite(tid=255, reqid=osd.11.0:255, at_version=82'106, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.870272+0000", 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.0238183850000002, 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0086404859999999993, 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870272+0000", 2024-09-16T06:49:14.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870272+0000", 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870271+0000", 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870333+0000", 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1748999999999994e-05 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870413+0000", 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0340999999999995e-05 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870433+0000", 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0029000000000001e-05 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871092+0000", 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00065900799999999999 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871113+0000", 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0968999999999999e-05 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878880+0000", 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007766935 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878913+0000", 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3039e-05 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 82/77 ECSubWrite(tid=188, reqid=osd.1.0:188, at_version=82'118, trim_to=0'0, roll_forward_to=82'114))", 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.871975+0000", 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.0221158240000001, 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068971400000000004, 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871975+0000", 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871975+0000", 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871946+0000", 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999709 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872054+0000", 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108447 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872176+0000", 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012151599999999999 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872199+0000", 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3479999999999999e-05 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872377+0000", 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017830799999999999 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872402+0000", 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4646000000000001e-05 2024-09-16T06:49:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878805+0000", 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064029430000000003 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878872+0000", 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7088999999999994e-05 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 82/81 ECSubWrite(tid=256, reqid=osd.11.0:256, at_version=82'108, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.873698+0000", 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.020393297, 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079502619999999996, 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873698+0000", 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873698+0000", 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873696+0000", 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873751+0000", 2024-09-16T06:49:14.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4925e-05 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873867+0000", 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115967 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873889+0000", 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2463000000000002e-05 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:14.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.875076+0000", 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011874089999999999 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.875100+0000", 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3142999999999998e-05 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.881595+0000", 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064951879999999998 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:14.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.881648+0000", 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3026999999999999e-05 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:14.986 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:14.989 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:14.989 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:49:14.990 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:14.990 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:14.990 INFO:tasks.rados.rados.0.smithi117.stdout:307: writing smithi11731613-23 from 3407872 to 4030464 tid 1 2024-09-16T06:49:14.990 INFO:tasks.rados.rados.0.smithi117.stdout:307: writing smithi11731613-23 from 4030464 to 4685824 tid 2 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: writing smithi11731613-23 from 4685824 to 5373952 tid 3 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: writing smithi11731613-23 from 5373952 to 5586944 tid 4 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:308: snap_create 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:305: finishing copy_from to smithi11731613-9 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:305: got expected ENOENT (src dne) 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:305: finishing copy_from racing read to smithi11731613-9 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: finishing write tid 1 to smithi11731613-23 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: finishing write tid 2 to smithi11731613-23 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: finishing write tid 3 to smithi11731613-23 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: finishing write tid 4 to smithi11731613-23 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: finishing write tid 5 to smithi11731613-23 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: finishing write tid 6 to smithi11731613-23 2024-09-16T06:49:14.991 INFO:tasks.rados.rados.0.smithi117.stdout:307: oid 23 updating version 0 to 125 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:307: oid 23 updating version 125 to 126 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:307: oid 23 version 126 is already newer than 123 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:307: oid 23 version 126 is already newer than 122 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:307: oid 23 version 126 is already newer than 124 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 23 v 126 (ObjNum 129 snap 32 seq_num 129) dirty exists 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:307: left oid 23 (ObjNum 129 snap 32 seq_num 129) 2024-09-16T06:49:14.992 INFO:tasks.rados.rados.0.smithi117.stdout:306: expect (ObjNum 123 snap 28 seq_num 123) 2024-09-16T06:49:14.992 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:14.992 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:14.982+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:15.089 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:15.087+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:15.115 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:15.115 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:15.130 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:49:15.190 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:15.192+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:15.231 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:15.231 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:15.249 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:49:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:49:15.249 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:15.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.8s4.5 2024-09-16T06:49:15.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:49:15.277 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:15.278 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:15.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:15.284+0000 7f85edc26640 0 -- 172.21.15.100:0/1166705113 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f85e821c0c0 msgr2=0x7f85e821e4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:15.291 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:15.286+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:15.393 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:15.390+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:7 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.528397304000002, 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023918795999999999, 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:49:15.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895879+0000", 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.895875+0000", 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896153+0000", 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000277688 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896194+0000", 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1916000000000003e-05 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896222+0000", 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.726e-05 2024-09-16T06:49:15.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896390+0000", 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016809299999999999 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896750+0000", 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036068999999999999 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898496+0000", 2024-09-16T06:49:15.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017459070000000001 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906280+0000", 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077831310000000004 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919636+0000", 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013356191 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919798+0000", 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016195000000000001 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:8 3.1s0 3:825f7e25:::smithi11731613-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.527746878000002, 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031459483000000003, 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.432 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 8 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896529+0000", 2024-09-16T06:49:15.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896527+0000", 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896557+0000", 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9323999999999999e-05 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896578+0000", 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1038000000000001e-05 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896594+0000", 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6038000000000001e-05 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900822+0000", 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042283219999999996 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901041+0000", 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021897799999999999 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902173+0000", 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00113226 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911105+0000", 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089313959999999994 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927760+0000", 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016655313000000001 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927989+0000", 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022873699999999999 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:9 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.527665849000002, 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.040864080999999997, 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 9 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896609+0000", 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896617+0000", 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9589999999999995e-06 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896634+0000", 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6872e-05 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896647+0000", 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3621999999999999e-05 2024-09-16T06:49:15.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902794+0000", 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061462820000000003 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903015+0000", 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221487 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928840+0000", 2024-09-16T06:49:15.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025824686999999999 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929046+0000", 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020604299999999999 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.937474+0000", 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084285460000000003 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:16 3.bs0 3:db3fccb6:::smithi11731613-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.527616238, 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024707216000000001, 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.443 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896660+0000", 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896659+0000", 2024-09-16T06:49:15.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897043+0000", 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038430400000000003 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897068+0000", 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5417000000000001e-05 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897087+0000", 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8218000000000001e-05 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897200+0000", 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011297499999999999 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897502+0000", 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030200600000000001 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899657+0000", 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021558250000000001 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911012+0000", 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011354401 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921200+0000", 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010188033000000001 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921367+0000", 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016735 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:17 3.bs0 3:db3fccb6:::smithi11731613-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.526389211000001, 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034791753000000002, 2024-09-16T06:49:15.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 17 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897885+0000", 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897900+0000", 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5515e-05 2024-09-16T06:49:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897935+0000", 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4703000000000003e-05 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897965+0000", 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9884000000000001e-05 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905007+0000", 2024-09-16T06:49:15.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007042296 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905241+0000", 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023388 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014152209999999999 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912065+0000", 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054085640000000003 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932510+0000", 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020444922000000001 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932679+0000", 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168894 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.453 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:18 3.bs0 3:db3fccb6:::smithi11731613-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.526230673999997, 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.041379560000000003, 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 18 2024-09-16T06:49:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898044+0000", 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898055+0000", 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0902e-05 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898077+0000", 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2395000000000002e-05 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898095+0000", 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7825e-05 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907329+0000", 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092342250000000004 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907544+0000", 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021447400000000001 2024-09-16T06:49:15.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932831+0000", 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025286896999999999 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933015+0000", 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018463 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939425+0000", 2024-09-16T06:49:15.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064098250000000001 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:37 3.bs0 3:d4957fab:::smithi11731613-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.519091469999999, 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029935371999999998, 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 37 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905185+0000", 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905182+0000", 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905624+0000", 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044199799999999998 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905656+0000", 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2393e-05 2024-09-16T06:49:15.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905676+0000", 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0285e-05 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907630+0000", 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019531459999999998 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:15.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907646+0000", 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6628000000000001e-05 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914043+0000", 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063962699999999999 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914333+0000", 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029089300000000002 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916429+0000", 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002095085 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923501+0000", 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007072404 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934947+0000", 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011445688000000001 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.935120+0000", 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017347200000000001 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:38 3.bs0 3:d4957fab:::smithi11731613-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.518578982999998, 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034120408999999997, 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 38 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905696+0000", 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905706+0000", 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9890000000000004e-06 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905725+0000", 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9341e-05 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905739+0000", 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4336e-05 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907725+0000", 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001985849 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907741+0000", 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5401e-05 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918115+0000", 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010374431999999999 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918349+0000", 2024-09-16T06:49:15.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023440900000000001 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919836+0000", 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014862639999999999 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.925093+0000", 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052577300000000004 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939675+0000", 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014581904999999999 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939818+0000", 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014229600000000001 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:39 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:15.471 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.518522855000001, 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039069524000000001, 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 39 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905752+0000", 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905759+0000", 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905776+0000", 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6694e-05 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905788+0000", 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2336e-05 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907813+0000", 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020251790000000002 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907828+0000", 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5094e-05 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921873+0000", 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014044235 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922092+0000", 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000219514 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940712+0000", 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018619988000000001 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940891+0000", 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017858199999999999 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.944823+0000", 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039321989999999999 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:40 3.7s0 3:e60b6e1b:::smithi11731613-14:head [append 0~229376,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.518474939999997, 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027254417, 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 40 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905801+0000", 2024-09-16T06:49:15.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905800+0000", 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906496+0000", 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00069684800000000002 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906524+0000", 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7344000000000001e-05 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906543+0000", 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9468000000000001e-05 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906717+0000", 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000173692 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906758+0000", 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1186000000000002e-05 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.913762+0000", 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070039339999999999 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914055+0000", 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029257000000000003 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916068+0000", 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020128730000000001 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921559+0000", 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054913260000000004 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932938+0000", 2024-09-16T06:49:15.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011379031 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933056+0000", 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117785 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:41 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.517698940000002, 2024-09-16T06:49:15.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027013423000000002, 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 41 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906577+0000", 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906575+0000", 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906603+0000", 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8030000000000001e-05 2024-09-16T06:49:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906622+0000", 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9008000000000002e-05 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906638+0000", 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5469e-05 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906985+0000", 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034709699999999999 2024-09-16T06:49:15.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906998+0000", 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3512e-05 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917405+0000", 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010406425 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917725+0000", 2024-09-16T06:49:15.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032018999999999998 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919697+0000", 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019721069999999999 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.924833+0000", 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051359170000000003 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933307+0000", 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084737270000000003 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933591+0000", 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028407699999999998 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:42 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.489 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.517620313999998, 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031605503, 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 42 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:15.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906654+0000", 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906661+0000", 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5719999999999999e-06 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906678+0000", 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7059e-05 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906698+0000", 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9551e-05 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907075+0000", 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037732500000000002 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907087+0000", 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1745e-05 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920576+0000", 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013489655 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.920843+0000", 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026655899999999998 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934288+0000", 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013445297 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.934560+0000", 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000272238 2024-09-16T06:49:15.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.938261+0000", 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037009289999999999 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:152 3.7s0 3:ebfb55a4:::smithi11731613-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.347868767000001, 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44514418, 2024-09-16T06:49:15.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 152 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076406+0000", 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076414+0000", 2024-09-16T06:49:15.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2889999999999998e-06 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076425+0000", 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0671e-05 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076433+0000", 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0870000000000007e-06 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079120+0000", 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026864800000000002 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079194+0000", 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4874000000000002e-05 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079523+0000", 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032904499999999998 2024-09-16T06:49:15.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080194+0000", 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067098400000000001 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521344+0000", 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.441149183 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521552+0000", 2024-09-16T06:49:15.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020799600000000001 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:153 3.7s0 3:ebfb55a4:::smithi11731613-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 52.347834941000002, 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44943085999999999, 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 153 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076440+0000", 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076444+0000", 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0559999999999998e-06 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076455+0000", 2024-09-16T06:49:15.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0789e-05 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076466+0000", 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0241000000000001e-05 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080491+0000", 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040256060000000001 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080588+0000", 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6849999999999996e-05 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522277+0000", 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44168930299999998 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522519+0000", 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241246 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.525872+0000", 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033535660000000001 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDPGPush(3.4s6 78/77 [PushOp(3:2ae27109:::smithi11731613-31:2a, version: 76'94, data_included: [786432~217088], data_size: 217088, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:2ae27109:::smithi11731613-31:2a@76'94, size: 4014080, copy_subset: [], clone_subset: {}, snapset: 2a=[]:{2a=[2a,29,28,24,22,21,20,1f,1d,1c,19,18,15]}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4014080, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.000631+0000", 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.4236456739999994, 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030189957, 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:15.506 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.000631+0000", 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.000631+0000", 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.000630+0000", 2024-09-16T06:49:15.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.000792+0000", 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016263000000000001 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.029941+0000", 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029148638000000001 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.029954+0000", 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3468999999999999e-05 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.030039+0000", 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4565000000000001e-05 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.030820+0000", 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00078147799999999995 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.509 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:694 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~4194304,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.001395+0000", 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.4228812650000009, 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032457092999999999, 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 694 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001395+0000", 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001395+0000", 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001394+0000", 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001401+0000", 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9140000000000008e-06 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001412+0000", 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0560000000000001e-05 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001418+0000", 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6210000000000001e-06 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001565+0000", 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014691800000000001 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003501+0000", 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019359539999999999 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.018337+0000", 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014836466 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.018976+0000", 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00063892599999999995 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.033852+0000", 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014876086 2024-09-16T06:49:15.514 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:695 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~4194304,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.001438+0000", 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.422838509, 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.036045328000000001, 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.515 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 695 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001438+0000", 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001438+0000", 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001437+0000", 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001440+0000", 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1e-06 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001445+0000", 2024-09-16T06:49:15.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8620000000000002e-06 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001449+0000", 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4790000000000001e-06 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003572+0000", 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021228150000000001 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-16T06:49:15.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003819+0000", 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024764200000000001 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.020118+0000", 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016299088 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.020504+0000", 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038530599999999997 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.037483+0000", 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016979448000000001 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:696 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~4194304,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.4227969530000006, 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038928996, 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 696 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:15.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001481+0000", 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3700000000000002e-06 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001486+0000", 2024-09-16T06:49:15.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2359999999999998e-06 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001489+0000", 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4199999999999999e-06 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003913+0000", 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024241890000000002 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-16T06:49:15.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.004188+0000", 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000274906 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.025189+0000", 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021001427 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.025394+0000", 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000204334 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.040408+0000", 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015014394 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpReadReply(3.1s0 78/77 ECSubReadReply(tid=45, attrs_read=0))", 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.004044+0000", 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.4202326089999993, 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.026329838000000001, 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:15.525 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.004044+0000", 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.004044+0000", 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.004043+0000", 2024-09-16T06:49:15.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.004267+0000", 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022419700000000001 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.029968+0000", 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025701742 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.029971+0000", 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.914e-06 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.030043+0000", 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1545999999999996e-05 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.030373+0000", 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033045200000000001 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:741 3.bs0 3:d4957fab:::smithi11731613-13:28 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.846644+0000", 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.5776322729999999, 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.036566170000000002, 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 741 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846644+0000", 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846644+0000", 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846642+0000", 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846652+0000", 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3729999999999999e-06 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846672+0000", 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0553000000000001e-05 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846687+0000", 2024-09-16T06:49:15.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5063e-05 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.847038+0000", 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035066999999999997 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.847053+0000", 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.448e-05 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:15.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.855965+0000", 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089126729999999994 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.856204+0000", 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023855099999999999 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.883210+0000", 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027006220000000001 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.533 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:15.534 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:15.534 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:15.535 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:15.535 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:15.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:49:15.536 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:15.533+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:15.539 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:15.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:15.636 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:15.635+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:15.664 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:15.665 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:15.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:49:15.737 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:15.740+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:15.837 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:15.838 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:15.838 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:15.837+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:15.852 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:49:15.889 INFO:tasks.rados.rados.0.smithi117.stdout:305: done (3 left) 2024-09-16T06:49:15.889 INFO:tasks.rados.rados.0.smithi117.stdout:306: done (2 left) 2024-09-16T06:49:15.890 INFO:tasks.rados.rados.0.smithi117.stdout:307: done (1 left) 2024-09-16T06:49:15.890 INFO:tasks.rados.rados.0.smithi117.stdout:308: done (0 left) 2024-09-16T06:49:15.890 INFO:tasks.rados.rados.0.smithi117.stdout:309: read oid 44 snap -1 2024-09-16T06:49:15.890 INFO:tasks.rados.rados.0.smithi117.stdout:309: expect (ObjNum 126 snap 31 seq_num 126) 2024-09-16T06:49:15.890 INFO:tasks.rados.rados.0.smithi117.stdout:310: delete oid 26 current snap is 33 2024-09-16T06:49:15.897 INFO:tasks.rados.rados.0.smithi117.stdout:310: done (1 left) 2024-09-16T06:49:15.898 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 41 current snap is 33 2024-09-16T06:49:15.898 INFO:tasks.rados.rados.0.smithi117.stdout:311: seq_num 131 ranges {4014080=655360,4669440=425984,5095424=638976,5734400=491520} 2024-09-16T06:49:15.938 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:15.935+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:15.942 INFO:tasks.rados.rados.0.smithi117.stdout:311: writing smithi11731613-41 from 4014080 to 4669440 tid 1 2024-09-16T06:49:15.947 INFO:tasks.rados.rados.0.smithi117.stdout:311: writing smithi11731613-41 from 4669440 to 5095424 tid 2 2024-09-16T06:49:15.954 INFO:tasks.rados.rados.0.smithi117.stdout:311: writing smithi11731613-41 from 5095424 to 5734400 tid 3 2024-09-16T06:49:15.959 INFO:tasks.rados.rados.0.smithi117.stdout:311: writing smithi11731613-41 from 5734400 to 6225920 tid 4 2024-09-16T06:49:15.959 INFO:tasks.rados.rados.0.smithi117.stdout:312: snap_remove snap 32 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=5, reqid=client.4511.0:59, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.041074408999997, 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063718670000000005, 2024-09-16T06:49:15.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941153+0000", 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941187+0000", 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3837000000000001e-05 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:15.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002113+0000", 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060926368000000002 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002134+0000", 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-05 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002230+0000", 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5789000000000001e-05 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002246+0000", 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5676e-05 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004821+0000", 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002575262 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004873+0000", 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2095000000000002e-05 2024-09-16T06:49:15.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=158, reqid=client.4511.0:77, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.006677371000002, 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051524863999999997, 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975550+0000", 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975589+0000", 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9523000000000001e-05 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021558+0000", 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045968459000000003 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021574+0000", 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6399999999999999e-05 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021656+0000", 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1742000000000004e-05 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021673+0000", 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7343e-05 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027012+0000", 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053383889999999998 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027076+0000", 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4325999999999994e-05 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.9s1 35/32 ECSubRead(tid=11, to_read={3:92c79891:::smithi11731613-34:head=0,4096,0}, subchunks={3:92c79891:::smithi11731613-34:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.970865854000003, 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034091585000000001, 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011362+0000", 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:15.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011374+0000", 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2490000000000001e-05 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044980+0000", 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033605818000000003 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044996+0000", 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5996999999999999e-05 2024-09-16T06:49:15.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045095+0000", 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9290000000000007e-05 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045454+0000", 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000359038 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:15.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.936637408999999, 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623068000000002, 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045590+0000", 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:15.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045617+0000", 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.724e-05 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073137+0000", 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027519538999999999 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073150+0000", 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3046e-05 2024-09-16T06:49:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073218+0000", 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7879999999999994e-05 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073228+0000", 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074167+0000", 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093834699999999997 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074214+0000", 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7722000000000002e-05 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.0s3 35/32 ECSubRead(tid=19, to_read={3:043b7e8c:::smithi11731613-16:head=0,65536,0}, subchunks={3:043b7e8c:::smithi11731613-16:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.900483643000001, 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.14017474999999999, 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081744+0000", 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081755+0000", 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0812000000000001e-05 2024-09-16T06:49:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221448+0000", 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.13969311000000001 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221465+0000", 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7030000000000001e-05 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221629+0000", 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163549 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221920+0000", 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000291181 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.5s3 41/37 ECSubRead(tid=7, to_read={3:af79444c:::smithi11731613-29:head=0,118784,0}, subchunks={3:af79444c:::smithi11731613-29:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.018459837000002, 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29065317299999999, 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963768+0000", 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963777+0000", 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2019999999999993e-06 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254192+0000", 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.290415064 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254204+0000", 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1430000000000001e-05 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254309+0000", 2024-09-16T06:49:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104763 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254422+0000", 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011342500000000001 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.125700618000003, 2024-09-16T06:49:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017061956, 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856527+0000", 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856580+0000", 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2326e-05 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856691+0000", 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011141600000000001 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856715+0000", 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3710000000000002e-05 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856826+0000", 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011109599999999999 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856852+0000", 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5686000000000001e-05 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873537+0000", 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016685114000000001 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873590+0000", 2024-09-16T06:49:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3242000000000003e-05 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 59/32 ECSubWrite(tid=54, reqid=client.4511.0:469, at_version=59'36, trim_to=0'0, roll_forward_to=57'34))", 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.141631428, 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91193373600000005, 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840907+0000", 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000310045 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749945+0000", 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.90903863699999998 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749964+0000", 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8073000000000001e-05 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750059+0000", 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5636000000000004e-05 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750079+0000", 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9893000000000001e-05 2024-09-16T06:49:16.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752481+0000", 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024015989999999999 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752531+0000", 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0263000000000003e-05 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=90, reqid=osd.1.0:90, at_version=61'54, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.165304145, 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018170465, 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816986+0000", 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1818999999999999e-05 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817088+0000", 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102377 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817111+0000", 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2946e-05 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817267+0000", 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015629899999999999 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817294+0000", 2024-09-16T06:49:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6412999999999999e-05 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835044+0000", 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017750359 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835095+0000", 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1004999999999997e-05 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.162260203999999, 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175777999999999, 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819967+0000", 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820019+0000", 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2649999999999999e-05 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820184+0000", 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016481499999999999 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820207+0000", 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.245e-05 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820328+0000", 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120942 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820352+0000", 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4732999999999999e-05 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835111+0000", 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014758378000000001 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835144+0000", 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3547000000000003e-05 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:16.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.150291268, 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016104653999999999, 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831994+0000", 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7025999999999999e-05 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832116+0000", 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012219299999999999 2024-09-16T06:49:16.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832140+0000", 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4023e-05 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832493+0000", 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035320700000000002 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832518+0000", 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5147000000000002e-05 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847982+0000", 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015463289 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848042+0000", 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0463e-05 2024-09-16T06:49:16.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.es4 71/37 ECSubWrite(tid=22, reqid=osd.8.0:22, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.147797397000001, 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019478348999999999, 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:16.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834430+0000", 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834474+0000", 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3739999999999998e-05 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834597+0000", 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122753 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834618+0000", 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1313000000000001e-05 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834923+0000", 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030448399999999999 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834948+0000", 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5293e-05 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853837+0000", 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018889153999999998 2024-09-16T06:49:16.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853910+0000", 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2368999999999998e-05 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=191, reqid=osd.11.0:191, at_version=71'82, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.146346711, 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175754, 2024-09-16T06:49:16.038 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835880+0000", 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835936+0000", 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6512999999999999e-05 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836143+0000", 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020638500000000001 2024-09-16T06:49:16.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836164+0000", 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1169999999999999e-05 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836644+0000", 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047988000000000002 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836664+0000", 2024-09-16T06:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0616999999999999e-05 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.851001+0000", 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014336389999999999 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.851058+0000", 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6969000000000003e-05 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=144, reqid=client.4511.0:542, at_version=71'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.141282964999998, 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016044704, 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840944+0000", 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841000+0000", 2024-09-16T06:49:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5287000000000001e-05 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841092+0000", 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2009000000000006e-05 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841121+0000", 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.885e-05 2024-09-16T06:49:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841388+0000", 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026696100000000002 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841425+0000", 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7626999999999997e-05 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856931+0000", 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015505755 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856990+0000", 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9401999999999998e-05 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.140704872000001, 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016485414, 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841522+0000", 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841612+0000", 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0575999999999995e-05 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841764+0000", 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015128200000000001 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841786+0000", 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1759999999999998e-05 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848074+0000", 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062882529999999997 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848099+0000", 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5510000000000001e-05 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857929+0000", 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098292880000000003 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858009+0000", 2024-09-16T06:49:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0599000000000005e-05 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=146, reqid=client.4511.0:543, at_version=71'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.139937204999999, 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018827678, 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842289+0000", 2024-09-16T06:49:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842343+0000", 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842426+0000", 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3009000000000004e-05 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842448+0000", 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1860000000000001e-05 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850355+0000", 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079069680000000003 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850377+0000", 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2167e-05 2024-09-16T06:49:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861052+0000", 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010674678 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861119+0000", 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7303000000000003e-05 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=147, reqid=client.4511.0:544, at_version=71'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.136300947999999, 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015229647000000001, 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845926+0000", 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845972+0000", 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5745999999999999e-05 2024-09-16T06:49:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846062+0000", 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9909999999999998e-05 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846133+0000", 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1109e-05 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853380+0000", 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072475899999999999 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853426+0000", 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5456e-05 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861126+0000", 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077004730000000002 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861157+0000", 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1130000000000002e-05 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=132, reqid=osd.1.0:132, at_version=72'83, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.138893813999999, 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015387283999999999, 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843334+0000", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844171+0000", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083714800000000002 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845132+0000", 2024-09-16T06:49:16.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096062600000000001 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845482+0000", 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035013699999999999 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845634+0000", 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015229600000000001 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845661+0000", 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858652+0000", 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012991745000000001 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858722+0000", 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9822999999999999e-05 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s6 72/37 ECSubWrite(tid=133, reqid=osd.1.0:133, at_version=72'36, trim_to=0'0, roll_forward_to=71'34))", 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.13804446, 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017512679999999999, 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844183+0000", 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844237+0000", 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4425999999999997e-05 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845784+0000", 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015470320000000001 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845809+0000", 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.455e-05 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845934+0000", 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012515900000000001 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845958+0000", 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3879999999999998e-05 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861640+0000", 2024-09-16T06:49:16.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01568172 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861697+0000", 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7173000000000003e-05 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=134, reqid=osd.1.0:134, at_version=72'85, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:16.070 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.137309662, 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017033627999999999, 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844918+0000", 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844980+0000", 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1920000000000003e-05 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846382+0000", 2024-09-16T06:49:16.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014028630000000001 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846403+0000", 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0143e-05 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848953+0000", 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002550667 2024-09-16T06:49:16.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848977+0000", 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3337e-05 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861911+0000", 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012934783 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861953+0000", 2024-09-16T06:49:16.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1192000000000001e-05 2024-09-16T06:49:16.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.075 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:16.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:49:16.077 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:16.080+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:16.177 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:16.174+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:16.244 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:16.245 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:16.245 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:16.245 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:16.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:49:16.277 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:16.280+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:16.378 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:16.378 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:16.379 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:16.378+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:16.392 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:49:16.393 INFO:teuthology.orchestra.run.smithi107.stdout:get_pg_num_history pg_num_history pg_num_history(e82 pg_nums {1={23=1},2={25=8},3={32=16}} deleted_pools ) 2024-09-16T06:49:16.393 INFO:teuthology.orchestra.run.smithi107.stdout:Importing pgid 3.8s4 2024-09-16T06:49:16.393 INFO:teuthology.orchestra.run.smithi107.stdout:Write 4#3:119b2380:::smithi11731613-26:15# 2024-09-16T06:49:16.393 INFO:teuthology.orchestra.run.smithi107.stdout:Write 4#3:119b2380:::smithi11731613-26:head# 2024-09-16T06:49:16.394 INFO:teuthology.orchestra.run.smithi107.stdout:snapset 2a=[]:{15=[15]} 2024-09-16T06:49:16.394 INFO:teuthology.orchestra.run.smithi107.stdout:Write 4#3:1cef29f0:::smithi11731613-40:24# 2024-09-16T06:49:16.394 INFO:teuthology.orchestra.run.smithi107.stdout:Write 4#3:1cef29f0:::smithi11731613-40:head# 2024-09-16T06:49:16.395 INFO:teuthology.orchestra.run.smithi107.stdout:snapset 24=[]:{24=[21,20,1f,1d,1c,19,18,15]} 2024-09-16T06:49:16.396 INFO:teuthology.orchestra.run.smithi107.stdout:write_pg epoch 82 info 3.8s4( v 82'38 (0'0,82'38] local-lis/les=32/33 n=4 ec=32/32 lis/c=32/32 les/c/f=33/33/0 sis=32) 2024-09-16T06:49:16.397 INFO:teuthology.orchestra.run.smithi107.stdout:Import successful 2024-09-16T06:49:16.479 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:16.474+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.862094776000006, 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:49:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:49:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:49:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.048294597999998, 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:49:16.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:49:16.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.048166046000006, 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:49:16.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:49:16.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.048128719000005, 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:49:16.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.047993720999997, 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:49:16.545 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:49:16.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:49:16.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:16.551 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.047961560000005, 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:49:16.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:49:16.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.557 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.047933129, 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:49:16.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:16.564 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.047826209999997, 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:49:16.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.354319699000001, 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.572 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:49:16.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.354289451, 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:49:16.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.354260015999998, 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:49:16.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:49:16.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:49:16.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.354149097000004, 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:49:16.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:49:16.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:77 3.8s0 3:119b2380:::smithi11731613-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:16.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.540979571000001, 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055956466000000003, 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 77 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971691+0000", 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971701+0000", 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8519999999999999e-06 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971715+0000", 2024-09-16T06:49:16.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4386000000000001e-05 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971725+0000", 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0360000000000001e-05 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.974948+0000", 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003222759 2024-09-16T06:49:16.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975027+0000", 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8497000000000004e-05 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975520+0000", 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049320500000000003 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.977378+0000", 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018576230000000001 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027491+0000", 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050112973999999998 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027649+0000", 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015816800000000001 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:78 3.8s0 3:119b2380:::smithi11731613-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.540884556999998, 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059293289999999998, 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.607 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971786+0000", 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971793+0000", 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971809+0000", 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5225000000000001e-05 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971820+0000", 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0922e-05 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975891+0000", 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040709759999999996 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:16.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.976067+0000", 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017645900000000001 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027711+0000", 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051643840000000003 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027855+0000", 2024-09-16T06:49:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014441399999999999 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031081+0000", 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032252359999999998 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 35/32 ECSubWriteReply(tid=162, last_complete=35'4, committed=1, applied=1))", 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.479919094000003, 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48350030599999999, 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032752+0000", 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032763+0000", 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1807000000000001e-05 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516055+0000", 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48329140799999998 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516068+0000", 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2782999999999999e-05 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516198+0000", 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129949 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516253+0000", 2024-09-16T06:49:16.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5390000000000003e-05 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=17, reqid=client.4511.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.45682283, 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.51123769299999999, 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:16.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055848+0000", 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055863+0000", 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.464e-05 2024-09-16T06:49:16.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565781+0000", 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50991880999999994 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565796+0000", 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4263e-05 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565932+0000", 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013609900000000001 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565949+0000", 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7229000000000001e-05 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567019+0000", 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001070463 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567087+0000", 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7229000000000003e-05 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.7s2 40/37 ECSubRead(tid=11, to_read={3:e60b6e1b:::smithi11731613-14:head=0,4096,0}, subchunks={3:e60b6e1b:::smithi11731613-14:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.445141434999996, 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.375282063, 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067529+0000", 2024-09-16T06:49:16.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067547+0000", 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8187000000000001e-05 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442125+0000", 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.37457846099999997 2024-09-16T06:49:16.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442142+0000", 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6843000000000001e-05 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442313+0000", 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017038200000000001 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442812+0000", 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049983999999999998 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:16.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.671108189999998, 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030302045, 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841562+0000", 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841613+0000", 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1739e-05 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841692+0000", 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8974000000000007e-05 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841718+0000", 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5452999999999998e-05 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847063+0000", 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053447629999999998 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847087+0000", 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4553999999999999e-05 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871791+0000", 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024703433 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871866+0000", 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5133e-05 2024-09-16T06:49:16.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s4 71/32 ECSubWrite(tid=63, reqid=osd.12.0:63, at_version=71'48, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.669444309999999, 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623895, 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843226+0000", 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843302+0000", 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5922999999999998e-05 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843397+0000", 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5536000000000001e-05 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843419+0000", 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2240000000000001e-05 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847145+0000", 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037257950000000001 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847167+0000", 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1590000000000002e-05 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:16.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871800+0000", 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024633057 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871851+0000", 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1647000000000002e-05 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:564 3.8s0 3:1cef29f0:::smithi11731613-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.537189305999998, 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020010844, 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:16.640 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 564 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:16.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975481+0000", 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975500+0000", 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975518+0000", 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:16.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975531+0000", 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3207000000000001e-05 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.983862+0000", 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083305979999999998 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.984001+0000", 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000139629 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990097+0000", 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060955419999999998 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:16.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990219+0000", 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122349 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.995493+0000", 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052739930000000003 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:16.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:16.647 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:16.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:49:16.649 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:16.645+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:16.748 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:16.746+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:16.821 INFO:tasks.rados.rados.0.smithi117.stdout:312: done (2 left) 2024-09-16T06:49:16.821 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 27 current snap is 33 2024-09-16T06:49:16.822 INFO:tasks.rados.rados.0.smithi117.stdout:313: seq_num 132 ranges {0=671744,671744=573440,1245184=753664,1998848=229376} 2024-09-16T06:49:16.823 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:16.823 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:16.823 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:16.823 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:16.828 INFO:tasks.rados.rados.0.smithi117.stdout:313: writing smithi11731613-27 from 0 to 671744 tid 1 2024-09-16T06:49:16.834 INFO:tasks.rados.rados.0.smithi117.stdout:313: writing smithi11731613-27 from 671744 to 1245184 tid 2 2024-09-16T06:49:16.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:49:16.843 INFO:tasks.rados.rados.0.smithi117.stdout:313: writing smithi11731613-27 from 1245184 to 1998848 tid 3 2024-09-16T06:49:16.846 INFO:tasks.rados.rados.0.smithi117.stdout:313: writing smithi11731613-27 from 1998848 to 2228224 tid 4 2024-09-16T06:49:16.846 INFO:tasks.rados.rados.0.smithi117.stdout:314: snap_remove snap 26 2024-09-16T06:49:16.850 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:16.849+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:16.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /var/log/ceph/exp.3.8s4.5 2024-09-16T06:49:16.951 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:16.950+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:16.989 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:49:16.999 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:16.999 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:17.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:49:17.051 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:17.049+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.241534633999997, 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022146409999999998, 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897264+0000", 2024-09-16T06:49:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897309+0000", 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5123000000000001e-05 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913004+0000", 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015694784999999999 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913046+0000", 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2945999999999998e-05 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914353+0000", 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306104 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914377+0000", 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4562999999999998e-05 2024-09-16T06:49:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919353+0000", 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049759360000000002 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919412+0000", 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9327000000000003e-05 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:22 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.239259091000001, 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023053033000000001, 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 22 2024-09-16T06:49:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899530+0000", 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901021+0000", 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014909019999999999 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901056+0000", 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5136999999999999e-05 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901077+0000", 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1622999999999999e-05 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901274+0000", 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019618500000000001 2024-09-16T06:49:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901606+0000", 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033268099999999998 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903794+0000", 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021876339999999999 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907586+0000", 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003792489 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922327+0000", 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014741124 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922595+0000", 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026710500000000001 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:23 3.3s0 3:ce1274e9:::smithi11731613-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.237695889000001, 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022719541999999999, 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901103+0000", 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901115+0000", 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1722999999999999e-05 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901134+0000", 2024-09-16T06:49:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9213000000000001e-05 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901149+0000", 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.523e-05 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905252+0000", 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041033170000000004 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905520+0000", 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026748400000000002 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.906744+0000", 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012242209999999999 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923321+0000", 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016576822000000001 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923623+0000", 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249399999999999 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923824+0000", 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000200955 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:24 3.3s0 3:ce1274e9:::smithi11731613-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.237637063000001, 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033126111999999999, 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901162+0000", 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901169+0000", 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901187+0000", 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8320000000000001e-05 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901203+0000", 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5872999999999999e-05 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907294+0000", 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006090949 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907512+0000", 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021778499999999999 2024-09-16T06:49:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924007+0000", 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016495668000000002 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924218+0000", 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021115199999999999 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934290+0000", 2024-09-16T06:49:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010071302000000001 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:31 3.3s0 3:c7654c4c:::smithi11731613-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.237584740000003, 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026931531000000002, 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901215+0000", 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902241+0000", 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010258190000000001 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902275+0000", 2024-09-16T06:49:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4037e-05 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902312+0000", 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.718e-05 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053663829999999997 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.908030+0000", 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035200800000000001 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912004+0000", 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039736770000000001 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923909+0000", 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011905429 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927953+0000", 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040441499999999998 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928147+0000", 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194005 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:32 3.3s0 3:c7654c4c:::smithi11731613-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.236468682000002, 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028234555000000001, 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 32 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902330+0000", 2024-09-16T06:49:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902341+0000", 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1279e-05 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902360+0000", 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902374+0000", 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3917000000000001e-05 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917230+0000", 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014855607 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917486+0000", 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025648799999999997 2024-09-16T06:49:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918644+0000", 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011572749999999999 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928363+0000", 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097195389999999993 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930283+0000", 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919505 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930566+0000", 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028369099999999998 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:33 3.3s0 3:c7654c4c:::smithi11731613-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.236413992000003, 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037765197, 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 33 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902386+0000", 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902392+0000", 2024-09-16T06:49:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893999999999998e-05 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902429+0000", 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8689999999999999e-05 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934639+0000", 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032210404999999998 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934843+0000", 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020376199999999999 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.940152+0000", 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053090200000000002 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=4, reqid=client.4511.0:44, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.221497306000003, 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788766000000001, 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917302+0000", 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917349+0000", 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7796000000000001e-05 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941504+0000", 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024154637 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941525+0000", 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0608999999999999e-05 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941640+0000", 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011531000000000001 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941662+0000", 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2328999999999999e-05 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943041+0000", 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001379047 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943092+0000", 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0834000000000002e-05 2024-09-16T06:49:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:68 3.3s0 3:cbbd057c:::smithi11731613-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.184775692000002, 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045964098000000002, 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 68 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954024+0000", 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954031+0000", 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954046+0000", 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954058+0000", 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1938e-05 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956544+0000", 2024-09-16T06:49:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024857949999999998 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956742+0000", 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019819 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.957650+0000", 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090843 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959725+0000", 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074256 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999914+0000", 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040189895000000003 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999989+0000", 2024-09-16T06:49:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4527e-05 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:69 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.184731206999999, 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048288992000000003, 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 69 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954068+0000", 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954074+0000", 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954088+0000", 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4866e-05 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954100+0000", 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1328999999999999e-05 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959103+0000", 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035749999999997 2024-09-16T06:49:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959283+0000", 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017953899999999999 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000081+0000", 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040797893000000002 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000261+0000", 2024-09-16T06:49:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180304 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002358+0000", 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002097392 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:98 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.143307341000003, 2024-09-16T06:49:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.086611296000000004, 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995492+0000", 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995501+0000", 2024-09-16T06:49:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8720000000000008e-06 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995517+0000", 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6650000000000002e-05 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995529+0000", 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1447999999999999e-05 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.997874+0000", 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023456250000000001 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998037+0000", 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016212600000000001 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998796+0000", 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075897099999999999 2024-09-16T06:49:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002377+0000", 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00358195 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081972+0000", 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.079594867 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082105+0000", 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013220899999999999 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:99 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.143240278, 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.087893389000000002, 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995559+0000", 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995565+0000", 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2659999999999998e-06 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995581+0000", 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.611e-05 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995594+0000", 2024-09-16T06:49:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2269000000000001e-05 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999116+0000", 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003522238 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999227+0000", 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111483 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082346+0000", 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.083118743999999994 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082416+0000", 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9993000000000006e-05 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.083454+0000", 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00103751 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:158 3.3s0 3:c7654c4c:::smithi11731613-11:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.059014468000001, 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41982487699999999, 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-16T06:49:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079784+0000", 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079794+0000", 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6260000000000007e-06 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079810+0000", 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6631000000000001e-05 2024-09-16T06:49:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079821+0000", 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0849000000000001e-05 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079917+0000", 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5730999999999995e-05 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.080126+0000", 2024-09-16T06:49:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020895399999999999 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081069+0000", 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094364700000000004 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082604+0000", 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001534863 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499525+0000", 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41692029400000002 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499611+0000", 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-05 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:226 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.183133667, 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027980046000000001, 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 226 2024-09-16T06:49:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955665+0000", 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955672+0000", 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000002e-06 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955690+0000", 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8179999999999999e-05 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955704+0000", 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.396e-05 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956020+0000", 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000316101 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956038+0000", 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8425999999999999e-05 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974108+0000", 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018069776999999999 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974338+0000", 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022928799999999999 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975586+0000", 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012484639999999999 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.979590+0000", 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040043939999999997 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983481+0000", 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038905480000000002 2024-09-16T06:49:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983647+0000", 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166045 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:348 3.3s0 3:c742dae1:::smithi11731613-33:head [append 0~622592] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.298111311, 2024-09-16T06:49:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020152403999999999, 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 348 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840687+0000", 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841715+0000", 2024-09-16T06:49:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010285260000000001 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841743+0000", 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7990000000000001e-05 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841758+0000", 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4686e-05 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842034+0000", 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027588800000000001 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842198+0000", 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000164599 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.843642+0000", 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014432760000000001 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.847983+0000", 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043409349999999998 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860716+0000", 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012732902000000001 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860842+0000", 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012617499999999999 2024-09-16T06:49:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:350 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (57) in=7b] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.296876443000002, 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019634848, 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 350 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841924+0000", 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841923+0000", 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841929+0000", 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6060000000000002e-06 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841938+0000", 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4970000000000007e-06 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841946+0000", 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9130000000000003e-06 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.848163+0000", 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062167339999999998 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.848319+0000", 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015559300000000001 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849250+0000", 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093117299999999998 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.853284+0000", 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040342399999999997 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.861427+0000", 2024-09-16T06:49:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081432659999999997 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.861559+0000", 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013169599999999999 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:351 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.296847286999999, 2024-09-16T06:49:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023956603, 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 351 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841957+0000", 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0199999999999996e-06 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841966+0000", 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-06 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841974+0000", 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.34e-06 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849799+0000", 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078258089999999995 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.850139+0000", 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033906799999999999 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862313+0000", 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012174318999999999 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862630+0000", 2024-09-16T06:49:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031676099999999998 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.865910+0000", 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032802859999999999 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 47/32 ECSubWrite(tid=68, reqid=client.4511.0:361, at_version=0'0, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.267283194999997, 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50083384399999997, 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871516+0000", 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871541+0000", 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5782000000000002e-05 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370610+0000", 2024-09-16T06:49:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.49906899100000002 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370625+0000", 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.429e-05 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370744+0000", 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119746 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370773+0000", 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8807e-05 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372302+0000", 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015286519999999999 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372351+0000", 2024-09-16T06:49:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9348999999999998e-05 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 49/32 ECSubWrite(tid=70, reqid=osd.3.0:70, at_version=49'51, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.283596023000001, 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023397583, 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855204+0000", 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967296 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855221+0000", 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877682+0000", 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022460635999999999 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877690+0000", 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5369999999999997e-06 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877755+0000", 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4584000000000004e-05 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877766+0000", 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0968000000000001e-05 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878577+0000", 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000811029 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878602+0000", 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5191e-05 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 73/32 ECSubWriteReply(tid=108, last_complete=73'90, committed=1, applied=1))", 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.269040672999999, 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071784013999999993, 2024-09-16T06:49:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869759+0000", 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869765+0000", 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7590000000000003e-06 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941447+0000", 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071682318999999994 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941458+0000", 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1099e-05 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941509+0000", 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1201e-05 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941544+0000", 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4538e-05 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.248 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:17.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:49:17.250 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:17.249+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:17.351 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:17.346+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:17.376 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:17.376 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:17.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:49:17.452 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:17.456+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:17.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:17.550 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:17.550 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:17.551 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:17.551 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:17.551 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:17.552 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:17.550+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:17.564 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:49:17.653 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:17.650+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:17.706 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:17.706 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:17.730 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:17.730 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:17.733 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:17.733 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:17.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:17.733 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:5 3.2s0 3:42d6f0b3:::smithi11731613-2:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.839009934000003, 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029483957000000002, 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892373+0000", 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892385+0000", 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1939999999999999e-05 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892403+0000", 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8202999999999998e-05 2024-09-16T06:49:17.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892421+0000", 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8189000000000002e-05 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895208+0000", 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786527 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895392+0000", 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000184255 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896349+0000", 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00095644999999999997 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902998+0000", 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066495039999999997 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921651+0000", 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018652716 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921859+0000", 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208359 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:6 3.2s0 3:42d6f0b3:::smithi11731613-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:17.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.838928754999998, 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034217268000000002, 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892455+0000", 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892464+0000", 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.67e-06 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892484+0000", 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0265999999999999e-05 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892500+0000", 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.596e-05 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897287+0000", 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047868049999999999 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897527+0000", 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023998800000000001 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922181+0000", 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024653615 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922389+0000", 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020808600000000001 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926674+0000", 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042850639999999999 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:43 3.2s0 3:4636348c:::smithi11731613-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.825710702000002, 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020346564000000001, 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905671+0000", 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909244+0000", 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035731700000000001 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909296+0000", 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2352999999999998e-05 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909321+0000", 2024-09-16T06:49:17.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4813000000000001e-05 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909751+0000", 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042994099999999998 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.910155+0000", 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040433999999999999 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912510+0000", 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002354929 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924742+0000", 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012231967 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.925839+0000", 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001096302 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926021+0000", 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000182492 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:44 3.2s0 3:4636348c:::smithi11731613-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:17.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.822025072000002, 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034540078000000002, 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909357+0000", 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909370+0000", 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2583e-05 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909390+0000", 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9930000000000001e-05 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909421+0000", 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1285999999999998e-05 2024-09-16T06:49:17.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913550+0000", 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041288399999999999 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913768+0000", 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000217691 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914791+0000", 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010230770000000001 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924746+0000", 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099553599999999999 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943736+0000", 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018990000999999999 2024-09-16T06:49:17.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943900+0000", 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016405 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:45 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.821950135999998, 2024-09-16T06:49:17.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040625472000000003, 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:17.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909434+0000", 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909441+0000", 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0330000000000003e-06 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909460+0000", 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8674000000000001e-05 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909473+0000", 2024-09-16T06:49:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3485999999999999e-05 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919590+0000", 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010117482000000001 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919812+0000", 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000221389 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944680+0000", 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024868369000000001 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944880+0000", 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019949900000000001 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.950061+0000", 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051807770000000001 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:230 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.775789035000003, 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0201987, 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 230 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955593+0000", 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955613+0000", 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0630000000000001e-05 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955662+0000", 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8702999999999998e-05 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955689+0000", 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.73e-05 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955849+0000", 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015934200000000001 2024-09-16T06:49:17.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955874+0000", 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.548e-05 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966673+0000", 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010799329999999999 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966857+0000", 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018410900000000001 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975795+0000", 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089374099999999998 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:231 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:17.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.775662525999998, 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021272859000000002, 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 231 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:17.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955721+0000", 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955733+0000", 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2129e-05 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955755+0000", 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1423000000000001e-05 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955770+0000", 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4790000000000001e-05 2024-09-16T06:49:17.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955974+0000", 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020403199999999999 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955982+0000", 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6940000000000001e-06 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.967793+0000", 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011811056 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.968023+0000", 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023004500000000001 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976996+0000", 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089721929999999998 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:232 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.775437981000003, 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022137836000000001, 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 232 2024-09-16T06:49:17.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955946+0000", 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955958+0000", 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-05 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955977+0000", 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.912e-05 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955993+0000", 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5894000000000001e-05 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956101+0000", 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107402 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956123+0000", 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2056999999999999e-05 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971700+0000", 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015577155000000001 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971931+0000", 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023091400000000001 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.978085+0000", 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061543179999999998 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 55/37 ECSubWrite(tid=121, reqid=client.4511.0:431, at_version=55'56, trim_to=0'0, roll_forward_to=52'54))", 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.913822887999999, 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.102329752, 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817619+0000", 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7778999999999997e-05 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917448+0000", 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.099828728000000005 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917466+0000", 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7907e-05 2024-09-16T06:49:17.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917613+0000", 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014708200000000001 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917643+0000", 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0315999999999999e-05 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919830+0000", 2024-09-16T06:49:17.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021864940000000002 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919892+0000", 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2076000000000007e-05 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 71/37 ECSubWrite(tid=190, reqid=osd.11.0:190, at_version=71'82, trim_to=0'0, roll_forward_to=65'80))", 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.832354+0000", 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.89903142, 2024-09-16T06:49:17.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017875147000000001, 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832354+0000", 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832354+0000", 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832353+0000", 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832404+0000", 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562999999999997e-05 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832509+0000", 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105066 2024-09-16T06:49:17.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832535+0000", 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5772999999999999e-05 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.833293+0000", 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075863500000000002 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.833317+0000", 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-05 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850149+0000", 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016832041999999998 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850229+0000", 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9561999999999995e-05 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 71/37 ECSubWrite(tid=192, reqid=osd.11.0:192, at_version=71'84, trim_to=0'0, roll_forward_to=65'80))", 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.894706569, 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022295459, 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836677+0000", 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836728+0000", 2024-09-16T06:49:17.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1388999999999999e-05 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836823+0000", 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4826000000000003e-05 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836845+0000", 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2310000000000002e-05 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836965+0000", 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119603 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836989+0000", 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4258999999999999e-05 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857456+0000", 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020466868999999999 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858974+0000", 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015182590000000001 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s5 71/32 ECSubWrite(tid=62, reqid=osd.12.0:62, at_version=71'46, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.838936+0000", 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.892449248999998, 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020004181999999999, 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.791 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838936+0000", 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838936+0000", 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838935+0000", 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838989+0000", 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3368999999999997e-05 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839096+0000", 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107113 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839119+0000", 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2887e-05 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839245+0000", 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012593500000000001 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839269+0000", 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4493000000000001e-05 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858875+0000", 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019606068000000001 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858940+0000", 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4962999999999994e-05 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.795 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s1 71/37 ECSubWrite(tid=144, reqid=client.4511.0:542, at_version=71'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.891033832000002, 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018959849000000001, 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:17.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840405+0000", 2024-09-16T06:49:17.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4206000000000002e-05 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840522+0000", 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116945 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840544+0000", 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2350000000000001e-05 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841221+0000", 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067698900000000002 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841246+0000", 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.497e-05 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.859269+0000", 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018022442999999999 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.859311+0000", 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2475999999999998e-05 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s5 71/32 ECSubWrite(tid=63, reqid=osd.12.0:63, at_version=71'48, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.843234+0000", 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.888151496999999, 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018047657000000002, 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843234+0000", 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843234+0000", 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843232+0000", 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843299+0000", 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7199000000000005e-05 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843410+0000", 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110893 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843432+0000", 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.19e-05 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850016+0000", 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065836469999999998 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850041+0000", 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5179999999999999e-05 2024-09-16T06:49:17.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861229+0000", 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011188600999999999 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861281+0000", 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1910000000000003e-05 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=71, reqid=client.4511.0:549, at_version=72'66, trim_to=0'0, roll_forward_to=72'64))", 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.833381060000001, 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051205101000000003, 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898003+0000", 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898034+0000", 2024-09-16T06:49:17.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1466e-05 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948063+0000", 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050028524999999997 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948079+0000", 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5869999999999999e-05 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948191+0000", 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112663 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948206+0000", 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.484e-05 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.949167+0000", 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096042199999999995 2024-09-16T06:49:17.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.949209+0000", 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2706000000000001e-05 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=77, reqid=client.4511.0:555, at_version=72'72, trim_to=0'0, roll_forward_to=72'70))", 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.76528076, 2024-09-16T06:49:17.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036435799999999997, 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966102+0000", 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966153+0000", 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0362999999999999e-05 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000163+0000", 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034010037999999999 2024-09-16T06:49:17.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000187+0000", 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4159e-05 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000326+0000", 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013869500000000001 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000356+0000", 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0511999999999999e-05 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.002480+0000", 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021234090000000001 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.002540+0000", 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0733999999999998e-05 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=82, reqid=client.4511.0:572, at_version=72'77, trim_to=0'0, roll_forward_to=72'72))", 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.719157806999998, 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.301395462, 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012226+0000", 2024-09-16T06:49:17.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012275+0000", 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8619999999999999e-05 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.302391+0000", 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29011663999999998 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.302400+0000", 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9409999999999996e-06 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.305565+0000", 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031649389999999999 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.305627+0000", 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1673000000000004e-05 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.313590+0000", 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079632660000000001 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.313623+0000", 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2855000000000003e-05 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 72/32 ECSubWriteReply(tid=114, last_complete=72'106, committed=1, applied=1))", 2024-09-16T06:49:17.818 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.406736531, 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.17086979499999999, 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:17.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324647+0000", 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324660+0000", 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2982999999999999e-05 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495344+0000", 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.17068462700000001 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495360+0000", 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5539000000000001e-05 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495453+0000", 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3367000000000002e-05 2024-09-16T06:49:17.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495518+0000", 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5278999999999996e-05 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:806 3.2s0 3:476f74d7:::smithi11731613-27:head [append 0~671744] snapc 35=[33,31,2e,2b,2a,28,21,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.90095013499999999, 2024-09-16T06:49:17.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022399347, 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 806 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830432+0000", 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831511+0000", 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010790019999999999 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831552+0000", 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1624e-05 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831570+0000", 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8056000000000001e-05 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831718+0000", 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014737600000000001 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831744+0000", 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6622999999999999e-05 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836450+0000", 2024-09-16T06:49:17.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047061350000000002 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836721+0000", 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027029199999999999 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.838621+0000", 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019002820000000001 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.841994+0000", 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003373029 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.852763+0000", 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010769065 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.852834+0000", 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.127e-05 2024-09-16T06:49:17.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:807 3.2s0 3:476f74d7:::smithi11731613-27:head [append 0~573440] snapc 35=[33,31,2e,2b,2a,28,21,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:16.836309+0000", 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.89507594700000004, 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018722806000000002, 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:17.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 807 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836309+0000", 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836309+0000", 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836308+0000", 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836842+0000", 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000534436 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836856+0000", 2024-09-16T06:49:17.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3905e-05 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836867+0000", 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0988999999999999e-05 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.840547+0000", 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003680114 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:17.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.840670+0000", 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012319699999999999 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.841393+0000", 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00072269499999999998 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.843655+0000", 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022617819999999999 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.854946+0000", 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011290826 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.855032+0000", 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6545999999999994e-05 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:17.834 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:17.836 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:17.836 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:314: done (3 left) 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:315: read oid 47 snap -1 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:315: expect (ObjNum 114 snap 26 seq_num 114) 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:316: read oid 36 snap -1 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:316: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 38 current snap is 33 2024-09-16T06:49:17.837 INFO:tasks.rados.rados.0.smithi117.stdout:309: expect (ObjNum 126 snap 31 seq_num 126) 2024-09-16T06:49:17.837 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:17.838 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:17.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:49:17.839 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:17.834+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:17.839 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:17.839 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:17.839 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:49:17.840 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:17.840 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:17.871 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:17.871 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:17.891 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:17.916 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:17.937 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:17.940+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:17.944 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:17.944 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:17.972 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:17.972 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:17.975 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:17.975 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:17.989 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:49:17.999 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:18.029 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:18.030 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:18.038 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:18.034+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:18.059 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:18.059 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:18.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:18.068+0000 7f1575d9b640 0 -- 172.21.15.100:0/3264496408 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f157821e160 msgr2=0x7f1578220550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:18.094 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:18.095 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:18.101 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:18.101 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:18.111 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:18.111 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:18.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:49:18.127 INFO:tasks.rados.rados.0.smithi117.stdout:311: finishing write tid 1 to smithi11731613-41 2024-09-16T06:49:18.127 INFO:tasks.rados.rados.0.smithi117.stdout:311: finishing write tid 2 to smithi11731613-41 2024-09-16T06:49:18.127 INFO:tasks.rados.rados.0.smithi117.stdout:311: finishing write tid 3 to smithi11731613-41 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: finishing write tid 4 to smithi11731613-41 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: finishing write tid 5 to smithi11731613-41 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: finishing write tid 6 to smithi11731613-41 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: oid 41 updating version 0 to 127 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: oid 41 version 127 is already newer than 124 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: oid 41 version 127 is already newer than 126 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: oid 41 updating version 127 to 128 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: oid 41 version 128 is already newer than 125 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 41 v 128 (ObjNum 131 snap 33 seq_num 131) dirty exists 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:311: left oid 41 (ObjNum 131 snap 33 seq_num 131) 2024-09-16T06:49:18.128 INFO:tasks.rados.rados.0.smithi117.stdout:313: finishing write tid 1 to smithi11731613-27 2024-09-16T06:49:18.129 INFO:tasks.rados.rados.0.smithi117.stdout:313: finishing write tid 2 to smithi11731613-27 2024-09-16T06:49:18.129 INFO:tasks.rados.rados.0.smithi117.stdout:313: finishing write tid 3 to smithi11731613-27 2024-09-16T06:49:18.129 INFO:tasks.rados.rados.0.smithi117.stdout:313: finishing write tid 4 to smithi11731613-27 2024-09-16T06:49:18.129 INFO:tasks.rados.rados.0.smithi117.stdout:317: seq_num 133 ranges {622592=688128,1310720=720896,2031616=475136,2506752=360448} 2024-09-16T06:49:18.137 INFO:tasks.rados.rados.0.smithi117.stdout:317: writing smithi11731613-38 from 622592 to 1310720 tid 1 2024-09-16T06:49:18.139 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:18.137+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:18.142 INFO:tasks.rados.rados.0.smithi117.stdout:317: writing smithi11731613-38 from 1310720 to 2031616 tid 2 2024-09-16T06:49:18.146 INFO:tasks.rados.rados.0.smithi117.stdout:317: writing smithi11731613-38 from 2031616 to 2506752 tid 3 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:317: writing smithi11731613-38 from 2506752 to 2867200 tid 4 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:309: done (5 left) 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:311: done (4 left) 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 34 current snap is 33 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:313: finishing write tid 5 to smithi11731613-27 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:313: finishing write tid 6 to smithi11731613-27 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:313: oid 27 updating version 0 to 145 2024-09-16T06:49:18.149 INFO:tasks.rados.rados.0.smithi117.stdout:313: oid 27 updating version 145 to 146 2024-09-16T06:49:18.150 INFO:tasks.rados.rados.0.smithi117.stdout:313: oid 27 version 146 is already newer than 142 2024-09-16T06:49:18.150 INFO:tasks.rados.rados.0.smithi117.stdout:313: oid 27 version 146 is already newer than 144 2024-09-16T06:49:18.150 INFO:tasks.rados.rados.0.smithi117.stdout:313: oid 27 version 146 is already newer than 143 2024-09-16T06:49:18.150 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 27 v 146 (ObjNum 132 snap 33 seq_num 132) dirty exists 2024-09-16T06:49:18.150 INFO:tasks.rados.rados.0.smithi117.stdout:313: left oid 27 (ObjNum 132 snap 33 seq_num 132) 2024-09-16T06:49:18.150 INFO:tasks.rados.rados.0.smithi117.stdout:316: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-16T06:49:18.163 INFO:tasks.rados.rados.0.smithi117.stdout:315: expect (ObjNum 114 snap 26 seq_num 114) 2024-09-16T06:49:18.240 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:18.241+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:18.246 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:18.247 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:18.260 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:49:18.345 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:18.342+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:18.380 INFO:tasks.rados.rados.0.smithi117.stdout:317: finishing write tid 1 to smithi11731613-38 2024-09-16T06:49:18.380 INFO:tasks.rados.rados.0.smithi117.stdout:317: finishing write tid 2 to smithi11731613-38 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: finishing write tid 3 to smithi11731613-38 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: finishing write tid 4 to smithi11731613-38 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: finishing write tid 5 to smithi11731613-38 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: finishing write tid 6 to smithi11731613-38 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: oid 38 updating version 0 to 53 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: oid 38 updating version 53 to 54 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: oid 38 updating version 54 to 57 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: oid 38 version 57 is already newer than 56 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: oid 38 version 57 is already newer than 55 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 38 v 57 (ObjNum 133 snap 33 seq_num 133) dirty exists 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:317: left oid 38 (ObjNum 133 snap 33 seq_num 133) 2024-09-16T06:49:18.381 INFO:tasks.rados.rados.0.smithi117.stdout:318: seq_num 134 ranges {6193152=704512,6897664=475136,7372800=573440,7946240=507904} 2024-09-16T06:49:18.429 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:18.429 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:18.443 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:49:18.443 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:18.444+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:18.463 INFO:tasks.rados.rados.0.smithi117.stdout:318: writing smithi11731613-34 from 6193152 to 6897664 tid 1 2024-09-16T06:49:18.466 INFO:tasks.rados.rados.0.smithi117.stdout:318: writing smithi11731613-34 from 6897664 to 7372800 tid 2 2024-09-16T06:49:18.471 INFO:tasks.rados.rados.0.smithi117.stdout:318: writing smithi11731613-34 from 7372800 to 7946240 tid 3 2024-09-16T06:49:18.475 INFO:tasks.rados.rados.0.smithi117.stdout:318: writing smithi11731613-34 from 7946240 to 8454144 tid 4 2024-09-16T06:49:18.475 INFO:tasks.rados.rados.0.smithi117.stdout:313: done (4 left) 2024-09-16T06:49:18.475 INFO:tasks.rados.rados.0.smithi117.stdout:315: done (3 left) 2024-09-16T06:49:18.475 INFO:tasks.rados.rados.0.smithi117.stdout:316: done (2 left) 2024-09-16T06:49:18.476 INFO:tasks.rados.rados.0.smithi117.stdout:317: done (1 left) 2024-09-16T06:49:18.476 INFO:tasks.rados.rados.0.smithi117.stdout:319: rmattr oid 8 current snap is 33 2024-09-16T06:49:18.476 INFO:tasks.rados.rados.0.smithi117.stdout:319: done (1 left) 2024-09-16T06:49:18.476 INFO:tasks.rados.rados.0.smithi117.stdout:320: copy_from oid 35 from oid 42 current snap is 33 2024-09-16T06:49:18.476 INFO:tasks.rados.rados.0.smithi117.stdout:318: finishing write tid 1 to smithi11731613-34 2024-09-16T06:49:18.476 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 44 current snap is 33 2024-09-16T06:49:18.477 INFO:tasks.rados.rados.0.smithi117.stdout:321: seq_num 135 ranges {5816320=720896,6537216=622592,7159808=688128,7847936=245760} 2024-09-16T06:49:18.513 INFO:tasks.rados.rados.0.smithi117.stdout:321: writing smithi11731613-44 from 5816320 to 6537216 tid 1 2024-09-16T06:49:18.517 INFO:tasks.rados.rados.0.smithi117.stdout:321: writing smithi11731613-44 from 6537216 to 7159808 tid 2 2024-09-16T06:49:18.521 INFO:tasks.rados.rados.0.smithi117.stdout:321: writing smithi11731613-44 from 7159808 to 7847936 tid 3 2024-09-16T06:49:18.523 INFO:tasks.rados.rados.0.smithi117.stdout:321: writing smithi11731613-44 from 7847936 to 8093696 tid 4 2024-09-16T06:49:18.523 INFO:tasks.rados.rados.0.smithi117.stdout:322: snap_remove snap 27 2024-09-16T06:49:18.543 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:18.541+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:18.584 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:18.584 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:18.598 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:49:18.643 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:18.642+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:18.744 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:18.738+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:18.763 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:18.763 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:18.777 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:49:18.845 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:18.842+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:18.910 INFO:tasks.rados.rados.0.smithi117.stdout:322: done (3 left) 2024-09-16T06:49:18.910 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 17 current snap is 33 2024-09-16T06:49:18.910 INFO:tasks.rados.rados.0.smithi117.stdout:318: finishing write tid 2 to smithi11731613-34 2024-09-16T06:49:18.910 INFO:tasks.rados.rados.0.smithi117.stdout:318: finishing write tid 3 to smithi11731613-34 2024-09-16T06:49:18.910 INFO:tasks.rados.rados.0.smithi117.stdout:318: finishing write tid 4 to smithi11731613-34 2024-09-16T06:49:18.910 INFO:tasks.rados.rados.0.smithi117.stdout:323: seq_num 136 ranges {1703936=737280,2441216=770048,3211264=540672,3751936=245760} 2024-09-16T06:49:18.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: writing smithi11731613-17 from 1703936 to 2441216 tid 1 2024-09-16T06:49:18.933 INFO:tasks.rados.rados.0.smithi117.stdout:323: writing smithi11731613-17 from 2441216 to 3211264 tid 2 2024-09-16T06:49:18.937 INFO:tasks.rados.rados.0.smithi117.stdout:323: writing smithi11731613-17 from 3211264 to 3751936 tid 3 2024-09-16T06:49:18.939 INFO:tasks.rados.rados.0.smithi117.stdout:323: writing smithi11731613-17 from 3751936 to 3997696 tid 4 2024-09-16T06:49:18.939 INFO:tasks.rados.rados.0.smithi117.stdout:324: snap_remove snap 21 2024-09-16T06:49:18.948 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:18.942+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:18.953 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:18.953 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:18.973 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:49:19.048 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:19.052+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:19.090 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.090 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.104 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:49:19.149 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:19.146+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:19.228 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.228 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:49:19.290 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:19.286+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:19.350 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:19.349+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:19.359 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.359 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.373 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:49:19.451 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:19.446+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:19.540 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.541 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.552 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:19.550+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:19.554 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:49:19.653 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:19.650+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:19.672 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.672 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:49:19.754 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:19.750+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:19.818 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.818 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:49:19.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:49:19.855 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:19.853+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:324: done (4 left) 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:325: snap_create 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: finishing write tid 5 to smithi11731613-34 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: finishing write tid 6 to smithi11731613-34 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: oid 34 updating version 0 to 42 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: oid 34 updating version 42 to 44 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: oid 34 version 44 is already newer than 43 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: oid 34 version 44 is already newer than 40 2024-09-16T06:49:19.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: oid 34 version 44 is already newer than 41 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 34 v 44 (ObjNum 134 snap 33 seq_num 134) dirty exists 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:318: left oid 34 (ObjNum 134 snap 33 seq_num 134) 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:320: finishing copy_from racing read to smithi11731613-35 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:320: finishing copy_from to smithi11731613-35 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 35 v 31 (ObjNum 82 snap 11 seq_num 82) dirty exists 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: finishing write tid 1 to smithi11731613-44 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: finishing write tid 2 to smithi11731613-44 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: finishing write tid 3 to smithi11731613-44 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: finishing write tid 4 to smithi11731613-44 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: finishing write tid 5 to smithi11731613-44 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: finishing write tid 6 to smithi11731613-44 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: oid 44 updating version 0 to 118 2024-09-16T06:49:19.926 INFO:tasks.rados.rados.0.smithi117.stdout:321: oid 44 updating version 118 to 122 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:321: oid 44 version 122 is already newer than 119 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:321: oid 44 version 122 is already newer than 120 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:321: oid 44 version 122 is already newer than 121 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 44 v 122 (ObjNum 135 snap 33 seq_num 135) dirty exists 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:321: left oid 44 (ObjNum 135 snap 33 seq_num 135) 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: finishing write tid 1 to smithi11731613-17 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: finishing write tid 2 to smithi11731613-17 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: finishing write tid 3 to smithi11731613-17 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: finishing write tid 4 to smithi11731613-17 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: finishing write tid 5 to smithi11731613-17 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: finishing write tid 6 to smithi11731613-17 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: oid 17 updating version 0 to 82 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: oid 17 version 82 is already newer than 79 2024-09-16T06:49:19.927 INFO:tasks.rados.rados.0.smithi117.stdout:323: oid 17 version 82 is already newer than 80 2024-09-16T06:49:19.928 INFO:tasks.rados.rados.0.smithi117.stdout:323: oid 17 updating version 82 to 83 2024-09-16T06:49:19.928 INFO:tasks.rados.rados.0.smithi117.stdout:323: oid 17 version 83 is already newer than 81 2024-09-16T06:49:19.928 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 17 v 83 (ObjNum 136 snap 33 seq_num 136) dirty exists 2024-09-16T06:49:19.928 INFO:tasks.rados.rados.0.smithi117.stdout:323: left oid 17 (ObjNum 136 snap 33 seq_num 136) 2024-09-16T06:49:19.956 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:19.957+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:19.959 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:19.959 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:19.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:49:20.057 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:20.050+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:20.096 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:20.097 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:20.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:20.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:49:20.158 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:20.157+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:20.252 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:20.252 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:20.258 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:20.260+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:20.271 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:20.271 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:20.278 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:20.278 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:20.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:49:20.303 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:20.304 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:20.328 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:20.328 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:20.350 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:20.350 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:20.351 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:20.359 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:20.358+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:20.376 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:20.376 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:20.403 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:20.404 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:20.418 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:20.418 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:20.418 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:20.418 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:20.419 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:20.429 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:20.429 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:20.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:49:20.457 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:20.457 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:20.487 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:20.487 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:20.487 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:20.492+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:20.515 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:20.515 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:20.541 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:20.541 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:20.560 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:20.554+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:20.570 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:20.571 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=12, reqid=client.4511.0:68, at_version=35'8, trim_to=0'0, roll_forward_to=35'6))", 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.619531058, 2024-09-16T06:49:20.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.041915173, 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961090+0000", 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961089+0000", 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.961108+0000", 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8260000000000001e-05 2024-09-16T06:49:20.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001032+0000", 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.039924492999999998 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001043+0000", 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1063000000000001e-05 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001095+0000", 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1703999999999998e-05 2024-09-16T06:49:20.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.001102+0000", 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8360000000000003e-06 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002963+0000", 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018614529999999999 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.003005+0000", 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2182999999999999e-05 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.584 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=14, reqid=client.4511.0:98, at_version=35'10, trim_to=0'0, roll_forward_to=35'7))", 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.578457305999997, 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082808355, 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002164+0000", 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002163+0000", 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002178+0000", 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4431e-05 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084421+0000", 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082243256000000001 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084430+0000", 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8880000000000003e-06 2024-09-16T06:49:20.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084484+0000", 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4193000000000001e-05 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084496+0000", 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1754e-05 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084946+0000", 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000450242 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084972+0000", 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6281000000000001e-05 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.3s4 35/32 ECSubWrite(tid=16, reqid=client.4511.0:158, at_version=35'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.496237176999998, 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41790576000000001, 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084384+0000", 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084383+0000", 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084401+0000", 2024-09-16T06:49:20.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7584999999999999e-05 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500734+0000", 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.41633377199999999 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500752+0000", 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7374e-05 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500862+0000", 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010999500000000001 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.500877+0000", 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5437000000000001e-05 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502236+0000", 2024-09-16T06:49:20.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001358389 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.502290+0000", 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4076000000000002e-05 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:252 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.517104426000003, 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.38343603199999998, 2024-09-16T06:49:20.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 252 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063517+0000", 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063515+0000", 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063523+0000", 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7430000000000002e-06 2024-09-16T06:49:20.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063545+0000", 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1310000000000001e-05 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.063562+0000", 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7629999999999999e-05 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068202+0000", 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046400269999999997 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.068467+0000", 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000264691 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070315+0000", 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018477190000000001 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.070443+0000", 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012795699999999999 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:28.446953+0000", 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.376510228 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:482 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~442368] snapc 1a=[1a,19,18,15,11,c,8,5] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 31.730294189999999, 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019730733, 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 482 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850327+0000", 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.850326+0000", 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851134+0000", 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00080810400000000003 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851148+0000", 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3689000000000001e-05 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851153+0000", 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8329999999999999e-06 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851191+0000", 2024-09-16T06:49:20.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8087000000000003e-05 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851262+0000", 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0964999999999996e-05 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.851984+0000", 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00072110499999999999 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.859456+0000", 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0074722319999999997 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.869982+0000", 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010525806 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.870058+0000", 2024-09-16T06:49:20.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6180000000000006e-05 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:511 3.7s0 3:e8f80969:::smithi11731613-49:head [append 0~442368] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:52.882250+0000", 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.698371313999999, 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.017244913000000001, 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.606 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 511 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882250+0000", 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882250+0000", 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882248+0000", 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882792+0000", 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00054419799999999997 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882812+0000", 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9802000000000001e-05 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882820+0000", 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2649999999999996e-06 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.882871+0000", 2024-09-16T06:49:20.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0850000000000003e-05 2024-09-16T06:49:20.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.883044+0000", 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000173031 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.883874+0000", 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000830089 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.885467+0000", 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001593115 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.899385+0000", 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013918216000000001 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.899495+0000", 2024-09-16T06:49:20.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000109623 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:512 3.7s0 3:e8f80969:::smithi11731613-49:head [append 0~442368] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:52.887502+0000", 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 27.693119417999998, 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015186880999999999, 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.615 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 512 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887502+0000", 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887502+0000", 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887501+0000", 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887896+0000", 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039566100000000001 2024-09-16T06:49:20.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887910+0000", 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4154999999999999e-05 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887919+0000", 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1240000000000005e-06 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.887958+0000", 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9740000000000002e-05 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.888046+0000", 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8164000000000005e-05 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.888736+0000", 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068976699999999998 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.890458+0000", 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001721658 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.902594+0000", 2024-09-16T06:49:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.012136378999999999 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:52.902689+0000", 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4430000000000002e-05 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:549 3.7s0 3:e60b6e1b:::smithi11731613-14:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.899393+0000", 2024-09-16T06:49:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.681228383000001, 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.055164967000000002, 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 549 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.622 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899393+0000", 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899393+0000", 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899392+0000", 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899397+0000", 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5399999999999997e-06 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899403+0000", 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7870000000000004e-06 2024-09-16T06:49:20.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899409+0000", 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1970000000000004e-06 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.900891+0000", 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001482627 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.900979+0000", 2024-09-16T06:49:20.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8354000000000001e-05 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.901408+0000", 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042863200000000003 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.902446+0000", 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001038244 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954441+0000", 2024-09-16T06:49:20.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.051994264999999998 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954558+0000", 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117141 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:550 3.7s0 3:e60b6e1b:::smithi11731613-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.899430+0000", 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.681191356999999, 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.057651908000000002, 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.627 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 550 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899430+0000", 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899430+0000", 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899429+0000", 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899453+0000", 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3285000000000002e-05 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899462+0000", 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0599999999999997e-06 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.899469+0000", 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.934e-06 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.902346+0000", 2024-09-16T06:49:20.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028773259999999999 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.902438+0000", 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1891999999999994e-05 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954593+0000", 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.052154813000000001 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.631 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.954665+0000", 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2537e-05 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957082+0000", 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024165699999999998 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.632 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:551 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~540672] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.948074+0000", 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.632546992999998, 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.017190535, 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 551 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948074+0000", 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948074+0000", 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948071+0000", 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948505+0000", 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00043384000000000001 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948537+0000", 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1826000000000003e-05 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948552+0000", 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948653+0000", 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010173199999999999 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.948828+0000", 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000174289 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950509+0000", 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016809049999999999 2024-09-16T06:49:20.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957393+0000", 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0068844889999999997 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.965043+0000", 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076495920000000002 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.965265+0000", 2024-09-16T06:49:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022198199999999999 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:555 3.7s0 3:e6e9cc11:::smithi11731613-37:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.958998+0000", 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.621623243999998, 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.047641277000000003, 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 555 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.958998+0000", 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.958998+0000", 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.958997+0000", 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959001+0000", 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0819999999999999e-06 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959005+0000", 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4660000000000001e-06 2024-09-16T06:49:20.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959009+0000", 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.985e-06 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.967578+0000", 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0085692860000000006 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.967871+0000", 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00029280099999999998 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.969068+0000", 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00119688 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.970919+0000", 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001850824 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006489+0000", 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.035569890999999999 2024-09-16T06:49:20.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006639+0000", 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000150538 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:556 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.959035+0000", 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.62158642, 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.052626567999999999, 2024-09-16T06:49:20.643 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 556 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959035+0000", 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959035+0000", 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959034+0000", 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959036+0000", 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9719999999999999e-06 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959041+0000", 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2479999999999998e-06 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.959044+0000", 2024-09-16T06:49:20.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2310000000000001e-06 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.969877+0000", 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010833746999999999 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.970197+0000", 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031932100000000001 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006695+0000", 2024-09-16T06:49:20.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.036498435000000003 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.006765+0000", 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0025999999999997e-05 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.011661+0000", 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004896063 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:569 3.7s0 3:e8f80969:::smithi11731613-49:head [append 0~573440] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.000711+0000", 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.579909981, 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015629909000000001, 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 569 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.000711+0000", 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.000711+0000", 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.000702+0000", 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999905 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001202+0000", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050003299999999999 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001230+0000", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7991999999999999e-05 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001244+0000", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3890000000000001e-05 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001306+0000", 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2623e-05 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.001432+0000", 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000125344 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.002725+0000", 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012930839999999999 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.004929+0000", 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002203881 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016214+0000", 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011285394000000001 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016341+0000", 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127055 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.653 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:572 3.7s0 3:e8f80969:::smithi11731613-49:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.571164756999998, 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.30836179800000002, 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 572 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.654 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009456+0000", 2024-09-16T06:49:20.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009463+0000", 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6830000000000008e-06 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009473+0000", 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0052e-05 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009482+0000", 2024-09-16T06:49:20.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9570000000000008e-06 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.014506+0000", 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050242560000000004 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.014686+0000", 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017992299999999999 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.015477+0000", 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00079010899999999999 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016467+0000", 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099052600000000004 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.317653+0000", 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.30118623999999999 2024-09-16T06:49:20.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.317818+0000", 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016493200000000001 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:573 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.571133012000001, 2024-09-16T06:49:20.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.31066434700000001, 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 573 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:20.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009488+0000", 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009491+0000", 2024-09-16T06:49:20.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1499999999999999e-06 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009501+0000", 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.006e-05 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.009508+0000", 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5920000000000003e-06 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.662 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016668+0000", 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071597359999999999 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.016752+0000", 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4153999999999994e-05 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.317902+0000", 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.30114975199999999 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.318063+0000", 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016121199999999999 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.320153+0000", 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020892990000000002 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:598 3.7s0 3:e60b6e1b:::smithi11731613-14:head [stat,delete] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.304558+0000", 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.276063575999999, 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.02248083, 2024-09-16T06:49:20.664 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 598 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304558+0000", 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304558+0000", 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304555+0000", 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304569+0000", 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4270000000000001e-05 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304602+0000", 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2725999999999998e-05 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304619+0000", 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6759999999999999e-05 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304707+0000", 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8028999999999994e-05 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.304878+0000", 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017184500000000001 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.305786+0000", 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00090726900000000004 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.307268+0000", 2024-09-16T06:49:20.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0014825280000000001 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.326936+0000", 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019667604000000002 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.327038+0000", 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102599 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:600 3.7s0 3:e6e9cc11:::smithi11731613-37:head [append 0~458752] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.581230+0000", 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.999391216999999, 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.015383671, 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 600 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581230+0000", 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581230+0000", 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581227+0000", 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581694+0000", 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046673599999999998 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581728+0000", 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4564000000000003e-05 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.581745+0000", 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6322999999999999e-05 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.582372+0000", 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000627051 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.582680+0000", 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030799100000000003 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.584495+0000", 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001815662 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587062+0000", 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002566774 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.596421+0000", 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0093590089999999997 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.596614+0000", 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019236500000000001 2024-09-16T06:49:20.674 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:602 3.7s0 3:e6e9cc11:::smithi11731613-37:head [setxattr _header (57) in=7b] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.587820+0000", 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.992801212, 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014926857, 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.675 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 602 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587820+0000", 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587820+0000", 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587819+0000", 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587826+0000", 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5870000000000004e-06 2024-09-16T06:49:20.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587836+0000", 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0295999999999999e-05 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587845+0000", 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2099999999999993e-06 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.593779+0000", 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059341139999999999 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.594040+0000", 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026153100000000001 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.595212+0000", 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001171657 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.601503+0000", 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062912419999999998 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.601701+0000", 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019831499999999999 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.602747+0000", 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010453719999999999 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.680 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:603 3.7s0 3:e6e9cc11:::smithi11731613-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.587875+0000", 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.992745963000001, 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.019344, 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 603 2024-09-16T06:49:20.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587875+0000", 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587875+0000", 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587874+0000", 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587879+0000", 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5129999999999998e-06 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587891+0000", 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1684e-05 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.587902+0000", 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1562999999999999e-05 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.597930+0000", 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010027968999999999 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.598199+0000", 2024-09-16T06:49:20.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000268857 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.602859+0000", 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046601280000000004 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.603117+0000", 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025791099999999999 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.607219+0000", 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041022020000000001 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 73/37 ECSubWriteReply(tid=95, last_complete=73'88, committed=1, applied=1))", 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.710101017, 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.083448052999999994, 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:20.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870520+0000", 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.870525+0000", 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3789999999999997e-06 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.952902+0000", 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.082376648999999996 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.952915+0000", 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3373000000000001e-05 2024-09-16T06:49:20.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.952993+0000", 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8119000000000002e-05 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.953968+0000", 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00097517899999999995 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.689 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:20.690 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:20.690 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:20.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:49:20.692 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:20.692 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:20.692 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:20.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:20.693 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:20.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:20.694 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:20.696+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:20.792 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:20.796+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:20.864 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:20.865 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:20.865 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:20.865 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:20.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:49:20.893 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:20.890+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:20.922 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:49:20.925 INFO:tasks.rados.rados.0.smithi117.stdout:318: done (4 left) 2024-09-16T06:49:20.925 INFO:tasks.rados.rados.0.smithi117.stdout:320: done (3 left) 2024-09-16T06:49:20.925 INFO:tasks.rados.rados.0.smithi117.stdout:321: done (2 left) 2024-09-16T06:49:20.926 INFO:tasks.rados.rados.0.smithi117.stdout:323: done (1 left) 2024-09-16T06:49:20.926 INFO:tasks.rados.rados.0.smithi117.stdout:325: done (0 left) 2024-09-16T06:49:20.926 INFO:tasks.rados.rados.0.smithi117.stdout:326: snap_remove snap 19 2024-09-16T06:49:20.994 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:20.990+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:21.039 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:21.039 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:21.039 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:21.039 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:21.039 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:21.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:49:21.099 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:21.104+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:52 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~294912,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.178 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.244971526, 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011921681999999999, 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 52 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.179 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936109+0000", 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936107+0000", 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936869+0000", 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076224400000000003 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936883+0000", 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3704e-05 2024-09-16T06:49:21.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936893+0000", 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0567000000000001e-05 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937221+0000", 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032796600000000001 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.937632+0000", 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00041103599999999999 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939883+0000", 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022507009999999999 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943370+0000", 2024-09-16T06:49:21.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034869190000000002 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.947880+0000", 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045105839999999998 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.948030+0000", 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015010800000000001 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:53 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.244170252000004, 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.679887028, 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 53 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936910+0000", 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936909+0000", 2024-09-16T06:49:21.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936915+0000", 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8130000000000001e-06 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936921+0000", 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9939999999999997e-06 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936925+0000", 2024-09-16T06:49:21.187 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9920000000000001e-06 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941426+0000", 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045008390000000004 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.188 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.941657+0000", 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023140500000000001 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.942737+0000", 2024-09-16T06:49:21.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001079637 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943730+0000", 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00099337199999999996 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616662+0000", 2024-09-16T06:49:21.190 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67293178399999998 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.616797+0000", 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135186 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.191 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:54 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.244152018000001, 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2642334100000001, 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 54 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.192 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936928+0000", 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936930+0000", 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.725e-06 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936933+0000", 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.698e-06 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.936967+0000", 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3618000000000002e-05 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943520+0000", 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065532869999999997 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943693+0000", 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017226899999999999 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617527+0000", 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.67383411699999995 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.617765+0000", 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023803499999999999 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201162+0000", 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58339697800000001 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.es0 35/32 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.562102162999999, 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58220860799999996, 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.198 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618978+0000", 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618977+0000", 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.618996+0000", 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9266000000000002e-05 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200456+0000", 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.58146017500000002 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200480+0000", 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3186000000000001e-05 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.200604+0000", 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012390599999999999 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:24.201187+0000", 2024-09-16T06:49:21.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00058315500000000002 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:433 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~737280] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.228705445999999, 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013805337000000001, 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.202 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 433 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952375+0000", 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.952372+0000", 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953751+0000", 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013786060000000001 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.204 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953786+0000", 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5605999999999998e-05 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953802+0000", 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5478999999999999e-05 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.953976+0000", 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000173979 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.954152+0000", 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017637499999999999 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.957075+0000", 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029234159999999999 2024-09-16T06:49:21.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.960418+0000", 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003342579 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965998+0000", 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055795719999999997 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966180+0000", 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000182578 2024-09-16T06:49:21.207 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:434 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~573440] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.219325468000001, 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011138353, 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 434 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.209 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961755+0000", 2024-09-16T06:49:21.210 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.210 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.210 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.210 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.961752+0000", 2024-09-16T06:49:21.210 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:21.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.211 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.211 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.211 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962728+0000", 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00097593799999999998 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962755+0000", 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6597e-05 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962769+0000", 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3583e-05 2024-09-16T06:49:21.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.962853+0000", 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4399000000000003e-05 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.963021+0000", 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000167625 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964326+0000", 2024-09-16T06:49:21.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013054519999999999 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972047+0000", 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007721042 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972718+0000", 2024-09-16T06:49:21.214 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00067065799999999997 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972893+0000", 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000175472 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:435 3.es0 3:7f6b1be4:::smithi11731613-18:head [append 0~196608] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.216104221999998, 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086272310000000008, 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 435 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964976+0000", 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.964973+0000", 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:21.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965356+0000", 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038219399999999998 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965388+0000", 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2276000000000001e-05 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965403+0000", 2024-09-16T06:49:21.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966329+0000", 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00092638200000000001 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.219 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.220 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966612+0000", 2024-09-16T06:49:21.220 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000282525 2024-09-16T06:49:21.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.220 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.968300+0000", 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016878450000000001 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973229+0000", 2024-09-16T06:49:21.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049290990000000002 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973498+0000", 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026954399999999998 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.973603+0000", 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105053 2024-09-16T06:49:21.222 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:436 3.es0 3:7f6b1be4:::smithi11731613-18:head [setxattr _header (57) in=7b] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.215664756000002, 2024-09-16T06:49:21.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.01058251, 2024-09-16T06:49:21.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.224 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.224 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 436 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.225 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965416+0000", 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965414+0000", 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965422+0000", 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.43e-06 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965435+0000", 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3332999999999999e-05 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965445+0000", 2024-09-16T06:49:21.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4800000000000007e-06 2024-09-16T06:49:21.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.969503+0000", 2024-09-16T06:49:21.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0040581259999999996 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.969750+0000", 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024736799999999998 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.970629+0000", 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087915900000000002 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975095+0000", 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044660719999999998 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975865+0000", 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076922299999999998 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.975998+0000", 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000133442 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.230 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:437 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.215626686, 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014457724, 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.231 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 437 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965454+0000", 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965453+0000", 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965457+0000", 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0500000000000002e-06 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965471+0000", 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4403000000000001e-05 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.965479+0000", 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7859999999999997e-06 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.974299+0000", 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0088199260000000005 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.974438+0000", 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000139519 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.976547+0000", 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021088449999999998 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.235 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.976785+0000", 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023817099999999999 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.979911+0000", 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031258570000000001 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 55/37 ECSubWriteReply(tid=6, last_complete=55'5, committed=1, applied=1))", 2024-09-16T06:49:21.236 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.214334483000002, 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061836670000000003, 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966746+0000", 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966745+0000", 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966753+0000", 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9759999999999995e-06 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966796+0000", 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3655999999999997e-05 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.966805+0000", 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1200000000000008e-06 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972502+0000", 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056967390000000001 2024-09-16T06:49:21.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:42.972929+0000", 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000427253 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 61/37 ECSubWrite(tid=39, reqid=osd.4.0:39, at_version=61'34, trim_to=0'0, roll_forward_to=58'32))", 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.355838657, 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059543850000000004, 2024-09-16T06:49:21.240 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825242+0000", 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825241+0000", 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825303+0000", 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1846000000000004e-05 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825467+0000", 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000164575 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.825495+0000", 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7705999999999999e-05 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.828570+0000", 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030753009999999999 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.828603+0000", 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2546999999999999e-05 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831167+0000", 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002564588 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831196+0000", 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8532000000000001e-05 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.244 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 61/54 ECSubWrite(tid=121, reqid=osd.3.0:121, at_version=61'51, trim_to=0'0, roll_forward_to=57'47))", 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.332399471999999, 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063343469999999997, 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:49:21.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848681+0000", 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848679+0000", 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848734+0000", 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5569999999999998e-05 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848826+0000", 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1281999999999998e-05 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848851+0000", 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4837e-05 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848916+0000", 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5440999999999999e-05 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.848926+0000", 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7359999999999996e-06 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854992+0000", 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060664559999999996 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.855015+0000", 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2955e-05 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.249 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/37 ECSubWriteReply(tid=16, last_complete=61'13, committed=1, applied=1))", 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.326223769000002, 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.246763658, 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:49:21.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854857+0000", 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854856+0000", 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.854862+0000", 2024-09-16T06:49:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7479999999999999e-06 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101354+0000", 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.24649253400000001 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101371+0000", 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7116e-05 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101507+0000", 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013554600000000001 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:49.101620+0000", 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011343199999999999 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:518 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1802240,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:54.894072+0000", 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.287008412999999, 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.024146883000000001, 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 518 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894072+0000", 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894072+0000", 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894068+0000", 2024-09-16T06:49:21.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894088+0000", 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0106999999999999e-05 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894125+0000", 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6508999999999999e-05 2024-09-16T06:49:21.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894143+0000", 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8476999999999999e-05 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894256+0000", 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112626 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op must wait for map", 2024-09-16T06:49:21.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894270+0000", 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4600000000000001e-05 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.897757+0000", 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00348702 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.897904+0000", 2024-09-16T06:49:21.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146539 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.918219+0000", 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.020314855 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:519 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1802240,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:54.894164+0000", 2024-09-16T06:49:21.259 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.286916723000001, 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.025821218, 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 519 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894164+0000", 2024-09-16T06:49:21.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894164+0000", 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894162+0000", 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894171+0000", 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0329999999999997e-06 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894191+0000", 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0046000000000001e-05 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894201+0000", 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6900000000000004e-06 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894425+0000", 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022422800000000001 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894444+0000", 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8377e-05 2024-09-16T06:49:21.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.898809+0000", 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043656099999999998 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.898985+0000", 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017558100000000001 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.919985+0000", 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.020999903 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:520 3.es0 3:7f6b1be4:::smithi11731613-18:head [read 0~1802240,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:54.894243+0000", 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.286837191, 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.027308196999999999, 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.265 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 520 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894243+0000", 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894243+0000", 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894241+0000", 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894252+0000", 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0319999999999999e-05 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894272+0000", 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0526e-05 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894288+0000", 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5644000000000001e-05 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894528+0000", 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000239764 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.894544+0000", 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6577000000000001e-05 2024-09-16T06:49:21.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.900957+0000", 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064123060000000004 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.901095+0000", 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013892900000000001 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:54.921551+0000", 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.020455804000000001 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 72/37 ECSubWrite(tid=72, reqid=client.4511.0:551, at_version=72'68, trim_to=0'0, roll_forward_to=72'65))", 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:59.950678+0000", 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.230402562999998, 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070099439999999997, 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.271 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950678+0000", 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950678+0000", 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950676+0000", 2024-09-16T06:49:21.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950795+0000", 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118702 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950897+0000", 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010257200000000001 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.950920+0000", 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2784000000000001e-05 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.951018+0000", 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7981999999999998e-05 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.951060+0000", 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1353000000000002e-05 2024-09-16T06:49:21.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957590+0000", 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065307220000000001 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:59.957688+0000", 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7372000000000002e-05 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.275 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.7s1 72/37 ECSubWrite(tid=81, reqid=client.4511.0:571, at_version=72'76, trim_to=0'0, roll_forward_to=72'72))", 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.013321+0000", 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.167759480000001, 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0069873089999999997, 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013321+0000", 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013321+0000", 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013319+0000", 2024-09-16T06:49:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013494+0000", 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017420099999999999 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013566+0000", 2024-09-16T06:49:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2434000000000005e-05 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013579+0000", 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3042e-05 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013631+0000", 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1907000000000003e-05 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.013638+0000", 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6000000000000003e-06 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.020275+0000", 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0066369819999999996 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.020308+0000", 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3531000000000001e-05 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds5 73/54 ECSubWrite(tid=184, reqid=osd.3.0:182, at_version=0'0, trim_to=0'0, roll_forward_to=73'67))", 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 20.311509948000001, 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058193519999999999, 2024-09-16T06:49:21.281 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869570+0000", 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869583+0000", 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2876e-05 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869667+0000", 2024-09-16T06:49:21.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4022000000000003e-05 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869677+0000", 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0681e-05 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869767+0000", 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9477999999999999e-05 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.869773+0000", 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0510000000000002e-06 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.875363+0000", 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055901900000000001 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:00.875390+0000", 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6588999999999999e-05 2024-09-16T06:49:21.285 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.as4 82/32 ECSubWrite(tid=221, reqid=osd.3.0:221, at_version=82'163, trim_to=0'0, roll_forward_to=81'159))", 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.863783+0000", 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.317296911, 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0095684470000000008, 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.863783+0000", 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.863783+0000", 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.863754+0000", 2024-09-16T06:49:21.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.99997 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.864482+0000", 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00072801400000000003 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.864787+0000", 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030507700000000001 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.865482+0000", 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00069567800000000003 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.865858+0000", 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037568299999999997 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.865881+0000", 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2887e-05 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.873307+0000", 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0074258340000000001 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.873352+0000", 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5089999999999997e-05 2024-09-16T06:49:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.291 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.291 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.291 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:21.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:49:21.293 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:21.296+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:21.393 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:21.386+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:21.462 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:21.462 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:21.462 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:21.462 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:21.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:49:21.519 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:21.524+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:21.594 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:21.593+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:21.598 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:21.598 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:21.598 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:21.598 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:21.598 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:21.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:49:21.695 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:21.693+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:48 3.0s0 3:043b7e8c:::smithi11731613-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.777 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.861500771000003, 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.016720260000000001, 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 48 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:49:21.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918855+0000", 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918854+0000", 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918860+0000", 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3960000000000003e-06 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918873+0000", 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2485e-05 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.918923+0000", 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0386999999999997e-05 2024-09-16T06:49:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920454+0000", 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015303949999999999 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.920536+0000", 2024-09-16T06:49:21.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2573999999999999e-05 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.933186+0000", 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.012650051000000001 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.933335+0000", 2024-09-16T06:49:21.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000148689 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.935575+0000", 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022402540000000001 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.783 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:59 3.0s0 3:0900ee20:::smithi11731613-20:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.840975802000003, 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.069099573999999997, 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 59 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939380+0000", 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939379+0000", 2024-09-16T06:49:21.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939383+0000", 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9040000000000002e-06 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939390+0000", 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939394+0000", 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9380000000000002e-06 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943863+0000", 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044688669999999996 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.943950+0000", 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7173e-05 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944394+0000", 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044454100000000001 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944887+0000", 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000492156 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008423+0000", 2024-09-16T06:49:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.063536255 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008479+0000", 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6462999999999998e-05 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:60 3.0s0 3:0900ee20:::smithi11731613-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:49:21.789 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.840913843000003, 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.076005592999999996, 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 60 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939442+0000", 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939441+0000", 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939445+0000", 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7340000000000002e-06 2024-09-16T06:49:21.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939451+0000", 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.241e-06 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.939457+0000", 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4580000000000001e-06 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944649+0000", 2024-09-16T06:49:21.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051924110000000001 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:22.944778+0000", 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012887900000000001 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.008904+0000", 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.064125946000000003 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.009068+0000", 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016377000000000001 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.015447+0000", 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063797940000000003 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:102 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:49:21.794 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.777990629999998, 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.046755945, 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 102 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002365+0000", 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002364+0000", 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002370+0000", 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5459999999999999e-06 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002385+0000", 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5724000000000001e-05 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.002394+0000", 2024-09-16T06:49:21.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3280000000000006e-06 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007411+0000", 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050172059999999997 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.007528+0000", 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117267 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014384+0000", 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006855809 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.014457+0000", 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2799999999999994e-05 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049121+0000", 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.034664014 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.799 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:128 3.0s0 3:03b32ae3:::smithi11731613-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.733883034000002, 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.031368511000000002, 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 128 2024-09-16T06:49:21.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046473+0000", 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046472+0000", 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046476+0000", 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.578e-06 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046485+0000", 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2289999999999992e-06 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046491+0000", 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6319999999999996e-06 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048421+0000", 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019297240000000001 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048515+0000", 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4073999999999994e-05 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.048890+0000", 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000375306 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.050760+0000", 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018703350000000001 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077776+0000", 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.027015092000000001 2024-09-16T06:49:21.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077841+0000", 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5504000000000002e-05 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:129 3.0s0 3:03b32ae3:::smithi11731613-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.733856013999997, 2024-09-16T06:49:21.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.036392558999999998, 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 129 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046500+0000", 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046499+0000", 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046502+0000", 2024-09-16T06:49:21.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8200000000000001e-06 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046507+0000", 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3510000000000001e-06 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.046512+0000", 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.464e-06 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049021+0000", 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025094789999999998 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.049099+0000", 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7998999999999996e-05 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077886+0000", 2024-09-16T06:49:21.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.028786875 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.077945+0000", 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8941999999999999e-05 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.082892+0000", 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049470909999999998 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:157 3.0s0 3:043b7e8c:::smithi11731613-16:head [read 0~262144,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.697206610999999, 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14327616900000001, 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 157 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083149+0000", 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083148+0000", 2024-09-16T06:49:21.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083152+0000", 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1920000000000001e-06 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083158+0000", 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1959999999999996e-06 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.083163+0000", 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2909999999999998e-06 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084618+0000", 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0014551430000000001 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.084738+0000", 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119295 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:23.226425+0000", 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14168755499999999 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.cs3 49/37 ECSubWrite(tid=98, reqid=osd.11.0:96, at_version=0'0, trim_to=0'0, roll_forward_to=49'26))", 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.925118853000001, 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14832383499999999, 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.815 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855237+0000", 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:36.855241+0000", 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5889999999999996e-06 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002739+0000", 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.14749786200000001 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002747+0000", 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5390000000000002e-06 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002811+0000", 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4498999999999994e-05 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.002824+0000", 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2883e-05 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.003527+0000", 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070348000000000003 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:37.003561+0000", 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3121999999999999e-05 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:674 3.9s0 3:92c79891:::smithi11731613-34:head [setxattr _header (57) in=7b] snapc 2a=[2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:03.908895+0000", 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 17.871460542000001, 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.37177128500000001, 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 674 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908895+0000", 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908895+0000", 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908894+0000", 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:21.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908899+0000", 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4129999999999999e-06 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908905+0000", 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.815e-06 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908911+0000", 2024-09-16T06:49:21.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9800000000000003e-06 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910067+0000", 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011550779999999999 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910153+0000", 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6929999999999996e-05 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910552+0000", 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039885300000000001 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.912889+0000", 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002336893 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280537+0000", 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.367647797 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280666+0000", 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012930999999999999 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:675 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e76)", 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:03.908918+0000", 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 17.871438025, 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.37382299800000002, 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.825 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.826 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.826 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.826 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.826 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 675 2024-09-16T06:49:21.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.827 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908918+0000", 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908918+0000", 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908917+0000", 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908920+0000", 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9289999999999998e-06 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908927+0000", 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4760000000000003e-06 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.908931+0000", 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1799999999999998e-06 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.910904+0000", 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019736509999999999 2024-09-16T06:49:21.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911021+0000", 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011655900000000001 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.281221+0000", 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.37019976599999999 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.281359+0000", 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138305 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.282741+0000", 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013815870000000001 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.832 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 76/32 ECSubWriteReply(tid=92, last_complete=76'28, committed=1, applied=1))", 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 17.868812520999999, 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.36963403, 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911543+0000", 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967296 2024-09-16T06:49:21.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:03.911544+0000", 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2869999999999999e-06 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280309+0000", 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.36876511000000001 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280325+0000", 2024-09-16T06:49:21.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5860000000000001e-05 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.280408+0000", 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2886999999999994e-05 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:04.281177+0000", 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00076903900000000001 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs1 82/77 ECSubWrite(tid=252, reqid=osd.11.0:252, at_version=82'107, trim_to=0'0, roll_forward_to=80'105))", 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.845024+0000", 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.935331551999999, 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.016453564, 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.845024+0000", 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.845024+0000", 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.845023+0000", 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.845073+0000", 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9264999999999997e-05 2024-09-16T06:49:21.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.847849+0000", 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027762870000000001 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.847864+0000", 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5324000000000001e-05 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.848036+0000", 2024-09-16T06:49:21.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017185799999999999 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.848060+0000", 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3533e-05 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.861424+0000", 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013364588 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.861478+0000", 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3359000000000002e-05 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.fs1 82/77 ECSubWrite(tid=253, reqid=osd.11.0:253, at_version=82'109, trim_to=0'0, roll_forward_to=80'105))", 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.847750+0000", 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.932605292, 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.016864800999999999, 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.847750+0000", 2024-09-16T06:49:21.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.847750+0000", 2024-09-16T06:49:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.847748+0000", 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.847802+0000", 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3415999999999998e-05 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.852716+0000", 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049144590000000004 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.852738+0000", 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2076999999999999e-05 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.854307+0000", 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001568533 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.854329+0000", 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1642999999999999e-05 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.864557+0000", 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.010228631 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.864615+0000", 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7958999999999999e-05 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:736 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~6193152,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.849107+0000", 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.931248287000001, 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.036161987999999999, 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.849 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 736 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849107+0000", 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849107+0000", 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849104+0000", 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849125+0000", 2024-09-16T06:49:21.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0905999999999999e-05 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849156+0000", 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1309999999999997e-05 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849177+0000", 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1246999999999998e-05 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849930+0000", 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075259999999999997 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.850164+0000", 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023419099999999999 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.885269+0000", 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.035105127 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:737 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~6193152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.849196+0000", 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.931159874, 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.043150613999999997, 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 737 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.855 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849196+0000", 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849196+0000", 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849194+0000", 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849204+0000", 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7529999999999996e-06 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849222+0000", 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7893e-05 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849243+0000", 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1546999999999999e-05 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.850871+0000", 2024-09-16T06:49:21.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016278760000000001 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.851005+0000", 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000133392 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.892346+0000", 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.041341580000000003 2024-09-16T06:49:21.859 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(client.4511.0:738 3.9s0 3:92c79891:::smithi11731613-34:head [read 0~6193152,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.849257+0000", 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.931098642, 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.64562484099999995, 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.860 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "client_info": { 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "client": "client.4511", 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "tid": 738 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849257+0000", 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849257+0000", 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849256+0000", 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849264+0000", 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9300000000000003e-06 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849280+0000", 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6252999999999998e-05 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.849293+0000", 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2894000000000001e-05 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.857620+0000", 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0083273329999999993 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.857900+0000", 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027973800000000002 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:10.494882+0000", 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.63698192600000003 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.9s0 82/32 ECSubReadReply(tid=105, attrs_read=0))", 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.862943+0000", 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.917412303000001, 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.022586447999999999, 2024-09-16T06:49:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.862943+0000", 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.862943+0000", 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.862942+0000", 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.871963+0000", 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090209209999999995 2024-09-16T06:49:21.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.873292+0000", 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001329249 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.873309+0000", 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7397e-05 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.873842+0000", 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053301900000000003 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.885530+0000", 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011687426000000001 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.9s0 82/32 ECSubReadReply(tid=107, attrs_read=0))", 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.863614+0000", 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.916741642, 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.63158932499999998, 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.863614+0000", 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.863614+0000", 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.863613+0000", 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.866681+0000", 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030673810000000001 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:10.489838+0000", 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.62315710499999999 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:10.489852+0000", 2024-09-16T06:49:21.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3957e-05 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:10.489991+0000", 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000139412 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:10.495203+0000", 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052119870000000004 2024-09-16T06:49:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpReadReply(3.9s0 82/32 ECSubReadReply(tid=106, attrs_read=0))", 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.867166+0000", 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.913190015, 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.025394068999999998, 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.867166+0000", 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.867166+0000", 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.867164+0000", 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.878283+0000", 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.011118665999999999 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.880491+0000", 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002208209 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.880509+0000", 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7733000000000001e-05 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.885980+0000", 2024-09-16T06:49:21.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054713549999999998 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.892560+0000", 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0065794130000000001 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.879 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s4 82/77 ECSubWrite(tid=91, reqid=osd.13.0:91, at_version=82'49, trim_to=0'0, roll_forward_to=81'47))", 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:09.868555+0000", 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 11.911800213999999, 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.024148517000000001, 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868555+0000", 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868555+0000", 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868555+0000", 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868591+0000", 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6269999999999997e-05 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868780+0000", 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018889800000000001 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:21.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868798+0000", 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7575999999999999e-05 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868932+0000", 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000134079 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.868954+0000", 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2387000000000001e-05 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.892662+0000", 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.023707694000000001 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:21.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:09.892704+0000", 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2138999999999998e-05 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:21.885 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:21.887 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:49:21.888 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:21.892+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:21.935 INFO:tasks.rados.rados.0.smithi117.stdout:326: done (0 left) 2024-09-16T06:49:21.935 INFO:tasks.rados.rados.0.smithi117.stdout:327: rollback oid 38 current snap is 34 2024-09-16T06:49:21.935 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 38 to 31 2024-09-16T06:49:21.935 INFO:tasks.rados.rados.0.smithi117.stdout:328: rollback oid 30 current snap is 34 2024-09-16T06:49:21.935 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 30 to 13 2024-09-16T06:49:21.935 INFO:tasks.rados.rados.0.smithi117.stdout:329: read oid 47 snap -1 2024-09-16T06:49:21.936 INFO:tasks.rados.rados.0.smithi117.stdout:329: expect (ObjNum 114 snap 26 seq_num 114) 2024-09-16T06:49:21.936 INFO:tasks.rados.rados.0.smithi117.stdout:330: snap_create 2024-09-16T06:49:21.963 INFO:tasks.rados.rados.0.smithi117.stdout:327: finishing rollback tid 0 to smithi11731613-38 2024-09-16T06:49:21.963 INFO:tasks.rados.rados.0.smithi117.stdout:327: finishing rollback tid 1 to smithi11731613-38 2024-09-16T06:49:21.965 INFO:tasks.rados.rados.0.smithi117.stdout:328: finishing rollback tid 0 to smithi11731613-30 2024-09-16T06:49:21.965 INFO:tasks.rados.rados.0.smithi117.stdout:328: finishing rollback tid 1 to smithi11731613-30 2024-09-16T06:49:21.965 INFO:tasks.rados.rados.0.smithi117.stdout:328: finishing rollback tid 2 to smithi11731613-30 2024-09-16T06:49:21.965 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 200 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:21.966 INFO:tasks.rados.rados.0.smithi117.stdout:327: finishing rollback tid 2 to smithi11731613-38 2024-09-16T06:49:21.966 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 38 v 71 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T06:49:21.987 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:21.992+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:21.992 INFO:tasks.rados.rados.0.smithi117.stdout:329: expect (ObjNum 114 snap 26 seq_num 114) 2024-09-16T06:49:22.002 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 13] out_osds: [] dead_osds: [5] live_osds: [0, 4, 8, 12, 1, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:49:22.002 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:49:22.002 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T06:49:22.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T06:49:22.032 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:22.032 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:22.032 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:49:22.032 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:22.049 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:49:22.088 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:22.082+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:22.170 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:22.170 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:22.170 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:49:22.170 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:49:22.170 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:22.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:49:22.229 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:22.229+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:22.289 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:22.289+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:22.306 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:22.306 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:49:22.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:49:22.306 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:10 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.410305624999999, 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025558260999999999, 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10 2024-09-16T06:49:22.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896135+0000", 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896131+0000", 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896384+0000", 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025249400000000003 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896423+0000", 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9551999999999997e-05 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896446+0000", 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2657999999999999e-05 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896601+0000", 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015511400000000001 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896938+0000", 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000336779 2024-09-16T06:49:22.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899233+0000", 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022945840000000001 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903156+0000", 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039230970000000004 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921526+0000", 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018370496 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921693+0000", 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000166916 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:11 3.4s0 3:2be2ffb2:::smithi11731613-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:22.312 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.409953021, 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025954171000000002, 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:49:22.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896487+0000", 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896485+0000", 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896497+0000", 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1669e-05 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896518+0000", 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1180999999999999e-05 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896534+0000", 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5732000000000001e-05 2024-09-16T06:49:22.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900249+0000", 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037148250000000002 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.900475+0000", 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022637300000000001 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901657+0000", 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001182249 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.909750+0000", 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0080923149999999992 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922220+0000", 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012470623 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922442+0000", 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022146399999999999 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:12 3.4s0 3:2be2ffb2:::smithi11731613-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.409851660999998, 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032662215000000001, 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 12 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896589+0000", 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896587+0000", 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896595+0000", 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9389999999999997e-06 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896616+0000", 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0571e-05 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896631+0000", 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5315000000000002e-05 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:49:22.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057790200000000002 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902594+0000", 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018354499999999999 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922598+0000", 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020004152000000001 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.923951+0000", 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001352771 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929251+0000", 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005300389 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:34 3.4s0 3:2123e90c:::smithi11731613-12:head [append 0~196608,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:22.323 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.405349381000001, 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031689311999999997, 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 34 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901091+0000", 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901089+0000", 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:22.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901956+0000", 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086772399999999995 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.901994+0000", 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7880000000000003e-05 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902023+0000", 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8479999999999998e-05 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902695+0000", 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067224400000000001 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902713+0000", 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7719000000000001e-05 2024-09-16T06:49:22.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.910786+0000", 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00807276 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.911114+0000", 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032852700000000001 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.914294+0000", 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003180113 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918604+0000", 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043102669999999996 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932586+0000", 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013981214 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932780+0000", 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019474 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:35 3.4s0 3:2123e90c:::smithi11731613-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.404394662000001, 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031230615999999999, 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 35 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902046+0000", 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902044+0000", 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902055+0000", 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0961999999999999e-05 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902075+0000", 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9748000000000001e-05 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902088+0000", 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3791999999999999e-05 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903130+0000", 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001041583 2024-09-16T06:49:22.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903146+0000", 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6511000000000002e-05 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.915912+0000", 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012765472999999999 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.916161+0000", 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024903700000000002 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.917305+0000", 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001143813 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.927287+0000", 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099818019999999997 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933082+0000", 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057955740000000004 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933276+0000", 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019429399999999999 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:36 3.4s0 3:2123e90c:::smithi11731613-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.403972602000003, 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038105068999999998, 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 36 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902468+0000", 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902465+0000", 2024-09-16T06:49:22.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902479+0000", 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3405e-05 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902529+0000", 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9638999999999997e-05 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902555+0000", 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903331+0000", 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077605999999999997 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903346+0000", 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5362999999999998e-05 2024-09-16T06:49:22.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918677+0000", 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015331205000000001 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919160+0000", 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00048271000000000002 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936255+0000", 2024-09-16T06:49:22.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017094597999999999 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.936424+0000", 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168789 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940573+0000", 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041493600000000004 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:228 3.4s0 3:2be2ffb2:::smithi11731613-4:head [rollback 5] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 54.348088085999997, 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015696132000000002, 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 228 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.343 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958352+0000", 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958355+0000", 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3280000000000002e-06 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958364+0000", 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6410000000000008e-06 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958370+0000", 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0009999999999998e-06 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958705+0000", 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033533 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958720+0000", 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4976e-05 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961715+0000", 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029947229999999999 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.961796+0000", 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1070999999999997e-05 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962213+0000", 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00041693299999999997 2024-09-16T06:49:22.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.964999+0000", 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027864159999999999 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974015+0000", 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090157589999999999 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974049+0000", 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3501999999999999e-05 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:229 3.4s0 3:2be2ffb2:::smithi11731613-4:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 54.347946379, 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015649126999999999, 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.349 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 229 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:49:22.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958494+0000", 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958497+0000", 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4709999999999999e-06 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958504+0000", 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1500000000000002e-06 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958510+0000", 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9399999999999999e-06 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958785+0000", 2024-09-16T06:49:22.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027471700000000001 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958790+0000", 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9470000000000002e-06 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962470+0000", 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036807070000000001 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962542+0000", 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1168999999999996e-05 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.962912+0000", 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037075200000000002 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965035+0000", 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021227419999999999 2024-09-16T06:49:22.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974112+0000", 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090767599999999997 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.974143+0000", 2024-09-16T06:49:22.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1315000000000001e-05 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:239 3.1s0 3:825f7e25:::smithi11731613-3:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 54.347874152999999, 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014095699999999999, 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.356 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 239 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:49:22.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958566+0000", 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958570+0000", 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7950000000000001e-06 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958576+0000", 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958583+0000", 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2629999999999997e-06 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958766+0000", 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018306299999999999 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.958783+0000", 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6801000000000001e-05 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963004+0000", 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004221168 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963078+0000", 2024-09-16T06:49:22.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3843000000000004e-05 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.963329+0000", 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025082499999999998 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.965054+0000", 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017253220000000001 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972634+0000", 2024-09-16T06:49:22.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075798469999999998 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:27.972662+0000", 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8232999999999999e-05 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:493 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.820729+0000", 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.485711893999998, 2024-09-16T06:49:22.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021206388999999999, 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 493 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820729+0000", 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820729+0000", 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820725+0000", 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820743+0000", 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7316000000000002e-05 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820778+0000", 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5271999999999997e-05 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820795+0000", 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7469000000000002e-05 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820976+0000", 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018091799999999999 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820989+0000", 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2758999999999999e-05 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.825101+0000", 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041114259999999996 2024-09-16T06:49:22.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.825255+0000", 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015408899999999999 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.841935+0000", 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016680326999999998 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:494 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.820816+0000", 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.485624278000003, 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025957694999999999, 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 494 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820816+0000", 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820816+0000", 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820815+0000", 2024-09-16T06:49:22.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820821+0000", 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8660000000000002e-06 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820832+0000", 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0429e-05 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820840+0000", 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1799999999999996e-06 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821142+0000", 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030235899999999998 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821167+0000", 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4868e-05 2024-09-16T06:49:22.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.826749+0000", 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055818980000000001 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.826925+0000", 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017605100000000001 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.846774+0000", 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019848973999999998 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:495 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:49.820881+0000", 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.485559268000003, 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.029233947999999999, 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.373 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 495 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820881+0000", 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820881+0000", 2024-09-16T06:49:22.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820880+0000", 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820886+0000", 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4199999999999998e-06 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820911+0000", 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4825e-05 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.820919+0000", 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2900000000000002e-06 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821210+0000", 2024-09-16T06:49:22.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000290916 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.821219+0000", 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7260000000000008e-06 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.831203+0000", 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099845200000000002 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.831476+0000", 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027300700000000002 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:49.850115+0000", 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018639038 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.378 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 64/37 ECSubWriteReply(tid=102, last_complete=64'64, committed=1, applied=1))", 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:51.863092+0000", 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.443348644, 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.95453391200000004, 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863092+0000", 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863092+0000", 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863091+0000", 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:51.863095+0000", 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6169999999999998e-06 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.815760+0000", 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.95266482200000002 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.815779+0000", 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9579e-05 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.815914+0000", 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134721 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.817626+0000", 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017117 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:501 3.4s0 3:2123e90c:::smithi11731613-12:head [copy-from ver 2] snapc 1f=[1f,1d,1c,1a,19,18,15,11,c,8] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T06:49:22.382 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:52.822239+0000", 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.484201264999999, 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014484334999999999, 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 501 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822239+0000", 2024-09-16T06:49:22.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822239+0000", 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822237+0000", 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822250+0000", 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3813e-05 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822284+0000", 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3575000000000003e-05 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822308+0000", 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3949000000000001e-05 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822403+0000", 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4914999999999995e-05 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.822582+0000", 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000179502 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.830939+0000", 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0083570699999999994 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.832645+0000", 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017055499999999999 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.836594+0000", 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039489670000000003 2024-09-16T06:49:22.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:52.836724+0000", 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000129677 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:566 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:59.975658+0000", 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.330782456000001, 2024-09-16T06:49:22.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015477143, 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 566 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975658+0000", 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975658+0000", 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975655+0000", 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975673+0000", 2024-09-16T06:49:22.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7105e-05 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975701+0000", 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8303e-05 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975720+0000", 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9236e-05 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975796+0000", 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6176999999999999e-05 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975913+0000", 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117197 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.991135+0000", 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015221718 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.392 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:567 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:59.975751+0000", 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.330689473, 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018008169000000001, 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 567 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975751+0000", 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975751+0000", 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975750+0000", 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975759+0000", 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8200000000000003e-06 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975778+0000", 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9831999999999999e-05 2024-09-16T06:49:22.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975796+0000", 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8102999999999999e-05 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.976579+0000", 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00078251799999999995 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.976664+0000", 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4647000000000005e-05 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.993759+0000", 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017095549000000002 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:568 3.4s0 3:2ae27109:::smithi11731613-31:head [read 0~4014080,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:22.397 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:59.975811+0000", 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.330629252000001, 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021129868, 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 568 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975811+0000", 2024-09-16T06:49:22.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975811+0000", 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975810+0000", 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975817+0000", 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8229999999999995e-06 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975835+0000", 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7399000000000001e-05 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.975850+0000", 2024-09-16T06:49:22.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5434000000000002e-05 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.977021+0000", 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001171168 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.977099+0000", 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8003000000000005e-05 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:59.996941+0000", 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019841630999999998 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:585 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~2834432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.288348+0000", 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.018092663000001, 2024-09-16T06:49:22.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015716081, 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 585 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288348+0000", 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288348+0000", 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288347+0000", 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288355+0000", 2024-09-16T06:49:22.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5639999999999999e-06 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288365+0000", 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8160000000000005e-06 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288371+0000", 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9569999999999999e-06 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.289105+0000", 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073468999999999997 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.289217+0000", 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011158 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.304064+0000", 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01484707 2024-09-16T06:49:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:586 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~2834432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.288377+0000", 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.018063659999999, 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017297056000000002, 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 586 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288377+0000", 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288377+0000", 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288376+0000", 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288380+0000", 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.929e-06 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288388+0000", 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2630000000000003e-06 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.288394+0000", 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2379999999999997e-06 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.291019+0000", 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026251030000000002 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.291161+0000", 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000142378 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.305674+0000", 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014512707 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.411 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:685 3.4s0 3:29582896:::smithi11731613-28:head [stat,delete] snapc 2b=[2b,2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:04.882836+0000", 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.423604829999999, 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029932010000001, 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 685 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.882836+0000", 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.882836+0000", 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.882833+0000", 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:22.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.882844+0000", 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0797e-05 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.882878+0000", 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3485999999999998e-05 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.882904+0000", 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6512e-05 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.886196+0000", 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032913550000000001 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:04.886203+0000", 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0369999999999997e-06 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.922125+0000", 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035922569 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for degraded object", 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.923348+0000", 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001223177 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.973346+0000", 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049997751999999999 2024-09-16T06:49:22.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.973498+0000", 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015170299999999999 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.974709+0000", 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012107509999999999 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.977320+0000", 2024-09-16T06:49:22.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002611941 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.985477+0000", 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081563539999999993 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:05.985829+0000", 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000351997 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.419 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.419 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.419 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:22.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 94, 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "b2976256-637c-4e28-94f6-99f20017bc5e", 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "created": "2024-09-16T06:47:53.157733+0000", 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "modified": "2024-09-16T06:49:21.924863+0000", 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "last_up_change": "2024-09-16T06:49:10.837870+0000", 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "last_in_change": "2024-09-16T06:49:04.862161+0000", 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_num": 5806080, 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_set": [ 2024-09-16T06:49:22.422 INFO:teuthology.orchestra.run.smithi100.stdout: "nodeep-scrub", 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "noscrub", 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "pglog_hardlimit", 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "purged_snapdirs", 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_deletes", 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "sortbitwise" 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_version": 6, 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_snapshot": "", 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_max": 3, 2024-09-16T06:49:22.423 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd": 16, 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "require_min_compat_client": "luminous", 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "min_compat_client": "jewel", 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "require_osd_release": "reef", 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_crimson": false, 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "pools": [ 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 1, 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": ".mgr", 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:05.838641+0000", 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:49:22.424 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 1, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 1, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 1, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 1, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 1, 2024-09-16T06:49:22.425 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "25", 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:49:22.426 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:49:22.427 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:49:22.428 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_max": 32, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 1 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:49:22.429 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": {} 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 15.380000114440918, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 15.380000114440918, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.12999999523162842, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 0.97786140441894531, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.430 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "rbd", 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:08.489111+0000", 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:49:22.431 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 8, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 8, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 8, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 8, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 8, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:49:22.432 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "28", 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 3, 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 28, 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:49:22.433 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-16T06:49:22.434 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:49:22.435 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd": {} 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 4, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 4, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.5, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 0.97786140441894531, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-16T06:49:22.436 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "unique_pool_0", 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-16T06:48:18.885917+0000", 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 8193, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 3, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 7, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 5, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 1, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-16T06:49:22.437 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 16, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 16, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 16, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 16, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 16, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-16T06:49:22.438 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "94", 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 59, 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 94, 2024-09-16T06:49:22.439 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T06:49:22.440 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "shecprofile", 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T06:49:22.441 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 16384, 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "options": {}, 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "rados": {} 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: "osds": [ 2024-09-16T06:49:22.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "54494294-f25f-41f2-bff7-106492665e07", 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.443 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6808", 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6809", 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6810", 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6811", 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.445 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6814", 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6815", 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.446 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6812", 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6813", 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1028275949 2024-09-16T06:49:22.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6809/1028275949", 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6811/1028275949", 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6815/1028275949", 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6813/1028275949", 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:49:22.448 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "a265e752-1cd6-497e-bcb9-c56591a5ce30", 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 77, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6808", 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6809", 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.450 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6811", 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6813", 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6819", 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6821", 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6815", 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6817", 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 930331367 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6809/930331367", 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6813/930331367", 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6821/930331367", 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6817/930331367", 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "dada34c6-4dc6-4f9f-bc31-85983fd2d7fd", 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.454 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 0, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6800", 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6801", 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6802", 2024-09-16T06:49:22.456 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6803", 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6806", 2024-09-16T06:49:22.457 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6807", 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.458 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6804", 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6805", 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4231506893 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6801/4231506893", 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6803/4231506893", 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6807/4231506893", 2024-09-16T06:49:22.459 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6805/4231506893", 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "6c0fc051-a4a0-4ce5-989a-668e75bbe3c0", 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.460 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 83, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6802", 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6803", 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6804", 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6805", 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6808", 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6809", 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6806", 2024-09-16T06:49:22.464 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6807", 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1995864012 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6803/1995864012", 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6805/1995864012", 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6809/1995864012", 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6807/1995864012", 2024-09-16T06:49:22.465 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "36a46387-b859-4937-bbd0-5dd708630c6e", 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.466 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.467 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6802", 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.468 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6803", 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6806", 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.469 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6807", 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6804", 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.470 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6805", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2054405220 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6801/2054405220", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6803/2054405220", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6807/2054405220", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6805/2054405220", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.471 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "073f3d23-77f0-4fd3-b067-27fa81137288", 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 0, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:49:22.472 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 83, 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6810", 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6812", 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.473 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6814", 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6816", 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.474 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6822", 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6823", 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.475 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6818", 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6820", 2024-09-16T06:49:22.476 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1081990569 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6812/1081990569", 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6816/1081990569", 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6823/1081990569", 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6820/1081990569", 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "exists" 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:49:22.477 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "8a81592f-3704-48f6-89ba-cc4997c2105e", 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 83, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6824", 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6825", 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6826", 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6827", 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.480 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6830", 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6831", 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6828", 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6829", 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1095047955 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.482 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6825/1095047955", 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6827/1095047955", 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6831/1095047955", 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6829/1095047955", 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27", 2024-09-16T06:49:22.483 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 0.645782470703125, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 37, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.484 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6818", 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6819", 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6820", 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6821", 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.486 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6824", 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6825", 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6822", 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6823", 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3607859516 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.488 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6819/3607859516", 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6821/3607859516", 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6825/3607859516", 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6823/3607859516", 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7ce38131-057f-4d45-88a5-dae8bcceae3c", 2024-09-16T06:49:22.489 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 83, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.490 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6816", 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6817", 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.491 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6818", 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6819", 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6822", 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6823", 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6820", 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6821", 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 232055141 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6817/232055141", 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6819/232055141", 2024-09-16T06:49:22.494 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6823/232055141", 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6821/232055141", 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "aa7fde6e-6563-4e37-8af2-9114d0091362", 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.495 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 25, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6824", 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6825", 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6826", 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6827", 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6830", 2024-09-16T06:49:22.498 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6831", 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.499 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6828", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6829", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1511881207 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6825/1511881207", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6827/1511881207", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6831/1511881207", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6829/1511881207", 2024-09-16T06:49:22.500 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "05c5389d-8082-4be3-aa6d-138e00720f06", 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.501 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6808", 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.502 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6809", 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6810", 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6811", 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6814", 2024-09-16T06:49:22.504 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6815", 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6812", 2024-09-16T06:49:22.505 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6813", 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 435900412 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6809/435900412", 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6811/435900412", 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6815/435900412", 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6813/435900412", 2024-09-16T06:49:22.506 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "99552590-1d0a-420e-b0c3-357246880658", 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.507 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 83, 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6810", 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6811", 2024-09-16T06:49:22.508 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6812", 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6813", 2024-09-16T06:49:22.509 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6816", 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.510 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6817", 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6814", 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.511 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6815", 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3208528083 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6811/3208528083", 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6813/3208528083", 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6817/3208528083", 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6815/3208528083", 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "165fea1c-74b6-4806-a0d9-47441e161ed5", 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 22, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.513 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6824", 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6825", 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6826", 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6827", 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6830", 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6831", 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.516 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6828", 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6829", 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3107910928 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.517 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6825/3107910928", 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6827/3107910928", 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6831/3107910928", 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6829/3107910928", 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "03402d0c-e7fd-4233-b2cd-bf79cd75e9bc", 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.518 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 21, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 77, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6800", 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6801", 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6802", 2024-09-16T06:49:22.520 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6803", 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6806", 2024-09-16T06:49:22.521 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6807", 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6804", 2024-09-16T06:49:22.522 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.107:6805", 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 316232366 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.107:6801/316232366", 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.107:6803/316232366", 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.107:6807/316232366", 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.107:6805/316232366", 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.523 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7b027faf-5c9e-4aee-9cb9-9c6459b8fe39", 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.524 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 32, 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6816", 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.525 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6817", 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6818", 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.526 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6819", 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6822", 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.527 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6823", 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6820", 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.528 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.115:6821", 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 693497656 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.115:6817/693497656", 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.115:6819/693497656", 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.115:6823/693497656", 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.115:6821/693497656", 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "4877aa40-c0d8-494f-8a98-36089ed694c2", 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 23, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 83, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-16T06:49:22.530 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6826", 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6827", 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.531 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6828", 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6829", 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.532 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6832", 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6833", 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.533 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6830", 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.117:6831", 2024-09-16T06:49:22.534 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 579912865 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.117:6827/579912865", 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.117:6829/579912865", 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.117:6833/579912865", 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.117:6831/579912865", 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_xinfo": [ 2024-09-16T06:49:22.535 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.584963+0000", 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.536 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.677023+0000", 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.537 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.608639+0000", 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.384489+0000", 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:17.579785+0000", 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.539 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.476690+0000", 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 83 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:21.476717+0000", 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:11.670056+0000", 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.541 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 8, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:19.668799+0000", 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 9, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.542 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.549502+0000", 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 10, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:04.194869+0000", 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 11, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:15.647481+0000", 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 12, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:49:15.898585+0000", 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 13, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.545 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.368452+0000", 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 14, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.600045+0000", 2024-09-16T06:49:22.546 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 15, 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540138322906710015, 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-16T06:48:03.567594+0000", 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap": [], 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_items": [], 2024-09-16T06:49:22.547 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_primaries": [], 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_temp": [], 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_temp": [], 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklist": {}, 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "range_blocklist": {}, 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profiles": { 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "default": { 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "2", 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "1", 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "jerasure", 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "reed_sol_van" 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.548 INFO:teuthology.orchestra.run.smithi100.stdout: "shecprofile": { 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "c": "2", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-device-class": "", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-root": "default", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "4", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "3", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "shecprofile", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "shec", 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "multiple" 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps_queue": [ 2024-09-16T06:49:22.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 31, 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 3 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 41, 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 4 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 53, 2024-09-16T06:49:22.550 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "new_removed_snaps": [ 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "snaps": [ 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "begin": 31, 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: "length": 1 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.551 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "new_purged_snaps": [], 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_node_flags": {}, 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "device_class_flags": {}, 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode": { 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_enabled": false, 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_bucket_count": 0, 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "degraded_stretch_mode": 0, 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "recovering_stretch_mode": 0, 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_bucket": 0 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:22.552 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:22.554 INFO:tasks.thrashosds.thrasher:j is {'epoch': 94, 'fsid': 'b2976256-637c-4e28-94f6-99f20017bc5e', 'created': '2024-09-16T06:47:53.157733+0000', 'modified': '2024-09-16T06:49:21.924863+0000', 'last_up_change': '2024-09-16T06:49:10.837870+0000', 'last_in_change': '2024-09-16T06:49:04.862161+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T06:48:05.838641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9778614044189453, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T06:48:08.489111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9778614044189453, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-16T06:48:18.885917+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '54494294-f25f-41f2-bff7-106492665e07', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6808', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6809', 'nonce': 1028275949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6810', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6811', 'nonce': 1028275949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6814', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6815', 'nonce': 1028275949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6812', 'nonce': 1028275949}, {'type': 'v1', 'addr': '172.21.15.100:6813', 'nonce': 1028275949}]}, 'public_addr': '172.21.15.100:6809/1028275949', 'cluster_addr': '172.21.15.100:6811/1028275949', 'heartbeat_back_addr': '172.21.15.100:6815/1028275949', 'heartbeat_front_addr': '172.21.15.100:6813/1028275949', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a265e752-1cd6-497e-bcb9-c56591a5ce30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 930331367}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6811', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 930331367}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6819', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 930331367}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6815', 'nonce': 930331367}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 930331367}]}, 'public_addr': '172.21.15.107:6809/930331367', 'cluster_addr': '172.21.15.107:6813/930331367', 'heartbeat_back_addr': '172.21.15.107:6821/930331367', 'heartbeat_front_addr': '172.21.15.107:6817/930331367', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'dada34c6-4dc6-4f9f-bc31-85983fd2d7fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 4231506893}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 4231506893}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 4231506893}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 4231506893}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 4231506893}]}, 'public_addr': '172.21.15.115:6801/4231506893', 'cluster_addr': '172.21.15.115:6803/4231506893', 'heartbeat_back_addr': '172.21.15.115:6807/4231506893', 'heartbeat_front_addr': '172.21.15.115:6805/4231506893', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6c0fc051-a4a0-4ce5-989a-668e75bbe3c0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6802', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6803', 'nonce': 1995864012}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6804', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6805', 'nonce': 1995864012}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6808', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6809', 'nonce': 1995864012}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6806', 'nonce': 1995864012}, {'type': 'v1', 'addr': '172.21.15.117:6807', 'nonce': 1995864012}]}, 'public_addr': '172.21.15.117:6803/1995864012', 'cluster_addr': '172.21.15.117:6805/1995864012', 'heartbeat_back_addr': '172.21.15.117:6809/1995864012', 'heartbeat_front_addr': '172.21.15.117:6807/1995864012', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '36a46387-b859-4937-bbd0-5dd708630c6e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 2054405220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6802', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6803', 'nonce': 2054405220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6806', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6807', 'nonce': 2054405220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6804', 'nonce': 2054405220}, {'type': 'v1', 'addr': '172.21.15.100:6805', 'nonce': 2054405220}]}, 'public_addr': '172.21.15.100:6801/2054405220', 'cluster_addr': '172.21.15.100:6803/2054405220', 'heartbeat_back_addr': '172.21.15.100:6807/2054405220', 'heartbeat_front_addr': '172.21.15.100:6805/2054405220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '073f3d23-77f0-4fd3-b067-27fa81137288', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 83, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6812', 'nonce': 1081990569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6816', 'nonce': 1081990569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 1081990569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 1081990569}, {'type': 'v1', 'addr': '172.21.15.107:6820', 'nonce': 1081990569}]}, 'public_addr': '172.21.15.107:6812/1081990569', 'cluster_addr': '172.21.15.107:6816/1081990569', 'heartbeat_back_addr': '172.21.15.107:6823/1081990569', 'heartbeat_front_addr': '172.21.15.107:6820/1081990569', 'state': ['exists']}, {'osd': 6, 'uuid': '8a81592f-3704-48f6-89ba-cc4997c2105e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 1095047955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1095047955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1095047955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1095047955}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1095047955}]}, 'public_addr': '172.21.15.115:6825/1095047955', 'cluster_addr': '172.21.15.115:6827/1095047955', 'heartbeat_back_addr': '172.21.15.115:6831/1095047955', 'heartbeat_front_addr': '172.21.15.115:6829/1095047955', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.645782470703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6818', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6819', 'nonce': 3607859516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6820', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6821', 'nonce': 3607859516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6824', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6825', 'nonce': 3607859516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6822', 'nonce': 3607859516}, {'type': 'v1', 'addr': '172.21.15.117:6823', 'nonce': 3607859516}]}, 'public_addr': '172.21.15.117:6819/3607859516', 'cluster_addr': '172.21.15.117:6821/3607859516', 'heartbeat_back_addr': '172.21.15.117:6825/3607859516', 'heartbeat_front_addr': '172.21.15.117:6823/3607859516', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '7ce38131-057f-4d45-88a5-dae8bcceae3c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6816', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6817', 'nonce': 232055141}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6818', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6819', 'nonce': 232055141}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6822', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6823', 'nonce': 232055141}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6820', 'nonce': 232055141}, {'type': 'v1', 'addr': '172.21.15.100:6821', 'nonce': 232055141}]}, 'public_addr': '172.21.15.100:6817/232055141', 'cluster_addr': '172.21.15.100:6819/232055141', 'heartbeat_back_addr': '172.21.15.100:6823/232055141', 'heartbeat_front_addr': '172.21.15.100:6821/232055141', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'aa7fde6e-6563-4e37-8af2-9114d0091362', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 1511881207}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 1511881207}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 1511881207}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 1511881207}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 1511881207}]}, 'public_addr': '172.21.15.107:6825/1511881207', 'cluster_addr': '172.21.15.107:6827/1511881207', 'heartbeat_back_addr': '172.21.15.107:6831/1511881207', 'heartbeat_front_addr': '172.21.15.107:6829/1511881207', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '05c5389d-8082-4be3-aa6d-138e00720f06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 435900412}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 435900412}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 435900412}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 435900412}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 435900412}]}, 'public_addr': '172.21.15.115:6809/435900412', 'cluster_addr': '172.21.15.115:6811/435900412', 'heartbeat_back_addr': '172.21.15.115:6815/435900412', 'heartbeat_front_addr': '172.21.15.115:6813/435900412', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '99552590-1d0a-420e-b0c3-357246880658', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6810', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6811', 'nonce': 3208528083}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6812', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6813', 'nonce': 3208528083}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6816', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6817', 'nonce': 3208528083}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6814', 'nonce': 3208528083}, {'type': 'v1', 'addr': '172.21.15.117:6815', 'nonce': 3208528083}]}, 'public_addr': '172.21.15.117:6811/3208528083', 'cluster_addr': '172.21.15.117:6813/3208528083', 'heartbeat_back_addr': '172.21.15.117:6817/3208528083', 'heartbeat_front_addr': '172.21.15.117:6815/3208528083', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '165fea1c-74b6-4806-a0d9-47441e161ed5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6824', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6825', 'nonce': 3107910928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6826', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6827', 'nonce': 3107910928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6830', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6831', 'nonce': 3107910928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6828', 'nonce': 3107910928}, {'type': 'v1', 'addr': '172.21.15.100:6829', 'nonce': 3107910928}]}, 'public_addr': '172.21.15.100:6825/3107910928', 'cluster_addr': '172.21.15.100:6827/3107910928', 'heartbeat_back_addr': '172.21.15.100:6831/3107910928', 'heartbeat_front_addr': '172.21.15.100:6829/3107910928', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '03402d0c-e7fd-4233-b2cd-bf79cd75e9bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 316232366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 316232366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 316232366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 316232366}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 316232366}]}, 'public_addr': '172.21.15.107:6801/316232366', 'cluster_addr': '172.21.15.107:6803/316232366', 'heartbeat_back_addr': '172.21.15.107:6807/316232366', 'heartbeat_front_addr': '172.21.15.107:6805/316232366', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '7b027faf-5c9e-4aee-9cb9-9c6459b8fe39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 693497656}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 693497656}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 693497656}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 693497656}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 693497656}]}, 'public_addr': '172.21.15.115:6817/693497656', 'cluster_addr': '172.21.15.115:6819/693497656', 'heartbeat_back_addr': '172.21.15.115:6823/693497656', 'heartbeat_front_addr': '172.21.15.115:6821/693497656', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '4877aa40-c0d8-494f-8a98-36089ed694c2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6826', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6827', 'nonce': 579912865}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6828', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6829', 'nonce': 579912865}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 579912865}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6830', 'nonce': 579912865}, {'type': 'v1', 'addr': '172.21.15.117:6831', 'nonce': 579912865}]}, 'public_addr': '172.21.15.117:6827/579912865', 'cluster_addr': '172.21.15.117:6829/579912865', 'heartbeat_back_addr': '172.21.15.117:6833/579912865', 'heartbeat_front_addr': '172.21.15.117:6831/579912865', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.584963+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.677023+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.608639+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.384489+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:17.579785+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.476690+0000', 'dead_epoch': 83}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:21.476717+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:11.670056+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:19.668799+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.549502+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:04.194869+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:15.647481+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:49:15.898585+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.368452+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.600045+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T06:48:03.567594+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 3}, {'begin': 41, 'length': 4}, {'begin': 53, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T06:49:22.554 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:49:22.554 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:22.424+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:22.556 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:22.550+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:22.658 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:22.654+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:22.686 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:22.686 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:22.686 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:49:22.687 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:22.694 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:22.701 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:49:22.758 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:22.760+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:22.819 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:22.819 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:22.819 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-16T06:49:22.819 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-16T06:49:22.819 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:22.833 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:49:22.839 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:22.840 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:22.859 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:22.854+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:22.862 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:22.862 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:22.881 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:22.882 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:49:22.887 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:22.887 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:22.899 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-16T06:49:21.930568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124043827,"num_objects":111,"num_object_clones":59,"num_object_copies":757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":8,"num_read":515,"num_read_kb":178471,"num_write":882,"num_write_kb":133606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":100876288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1525,"ondisk_log_size":1525,"up":129,"acting":129,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":706660,"kb_used_data":225120,"kb_used_omap":24,"kb_used_meta":481255,"kb_avail":1509242780,"statfs":{"total":1546188226560,"available":1545464606720,"internally_reserved":0,"allocated":230522880,"data_stored":344301767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":492805216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8470528,"num_objects":2,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":1732,"num_write":16,"num_write_kb":8272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.099996"},"pg_stats":[{"pgid":"3.f","version":"89'130","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T06:49:16.855085+0000","last_change":"2024-09-16T06:49:16.856771+0000","last_active":"2024-09-16T06:49:16.855085+0000","last_peered":"2024-09-16T06:49:16.855085+0000","last_clean":"2024-09-16T06:49:16.855085+0000","last_became_active":"2024-09-16T06:49:05.959106+0000","last_became_peered":"2024-09-16T06:49:05.959106+0000","last_unstale":"2024-09-16T06:49:16.855085+0000","last_undegraded":"2024-09-16T06:49:16.855085+0000","last_fullsized":"2024-09-16T06:49:16.855085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:21.893464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012782605000000001,"stat_sum":{"num_bytes":10272768,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":32,"num_read_kb":4479,"num_write":47,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":7307264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,2,13,1],"acting":[11,12,4,14,2,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"20","length":"1"},{"start":"29","length":"1"},{"start":"2c","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"86'32","reported_seq":229,"reported_epoch":92,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:19.934858+0000","last_change":"2024-09-16T06:49:12.265575+0000","last_active":"2024-09-16T06:49:19.934858+0000","last_peered":"2024-09-16T06:49:19.934858+0000","last_clean":"2024-09-16T06:49:09.884100+0000","last_became_active":"2024-09-16T06:49:12.265575+0000","last_became_peered":"2024-09-16T06:49:12.265575+0000","last_unstale":"2024-09-16T06:49:19.934858+0000","last_undegraded":"2024-09-16T06:49:11.860617+0000","last_fullsized":"2024-09-16T06:49:11.860421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":7,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:29.453543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018081014999999999,"stat_sum":{"num_bytes":3735552,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":5284,"num_write":30,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":6127616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,2147483647],"acting":[8,14,6,11,13,0,2147483647],"avail_no_missing":["8(0)","0(5)","6(2)","11(3)","13(4)","14(1)"],"object_location_counts":[{"shards":"0(5),6(2),8(0),11(3),13(4),14(1)","objects":4}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.d","version":"91'83","reported_seq":305,"reported_epoch":91,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:18.963920+0000","last_change":"2024-09-16T06:49:11.864865+0000","last_active":"2024-09-16T06:49:18.963920+0000","last_peered":"2024-09-16T06:49:18.963920+0000","last_clean":"2024-09-16T06:49:09.863855+0000","last_became_active":"2024-09-16T06:49:11.864865+0000","last_became_peered":"2024-09-16T06:49:11.864865+0000","last_unstale":"2024-09-16T06:49:18.963920+0000","last_undegraded":"2024-09-16T06:49:11.858666+0000","last_fullsized":"2024-09-16T06:49:11.858254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":6,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:32.476279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019226413000000001,"stat_sum":{"num_bytes":8650752,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":5673,"num_write":33,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7372800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,2147483647],"acting":[7,3,0,12,9,13,2147483647],"avail_no_missing":["3(1)","0(2)","7(0)","9(4)","12(3)","13(5)"],"object_location_counts":[{"shards":"0(2),3(1),7(0),9(4),12(3),13(5)","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"85'116","reported_seq":340,"reported_epoch":89,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:16.843087+0000","last_change":"2024-09-16T06:49:11.863339+0000","last_active":"2024-09-16T06:49:16.843087+0000","last_peered":"2024-09-16T06:49:16.843087+0000","last_clean":"2024-09-16T06:49:09.898987+0000","last_became_active":"2024-09-16T06:49:11.863339+0000","last_became_peered":"2024-09-16T06:49:11.863339+0000","last_unstale":"2024-09-16T06:49:16.843087+0000","last_undegraded":"2024-09-16T06:49:11.858719+0000","last_fullsized":"2024-09-16T06:49:11.858364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.861414+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.034010460999999999,"stat_sum":{"num_bytes":13877248,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":24,"num_read_kb":20989,"num_write":44,"num_write_kb":15648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":25968640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,2147483647,12,4,1,9],"acting":[11,13,2147483647,12,4,1,9],"avail_no_missing":["11(0)","1(5)","4(4)","9(6)","12(3)","13(1)"],"object_location_counts":[{"shards":"1(5),4(4),9(6),11(0),12(3),13(1)","objects":7}],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'27","reported_seq":90,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.939632+0000","last_change":"2024-09-16T06:49:19.941210+0000","last_active":"2024-09-16T06:49:19.939632+0000","last_peered":"2024-09-16T06:49:19.939632+0000","last_clean":"2024-09-16T06:49:19.939632+0000","last_became_active":"2024-09-16T06:49:05.925903+0000","last_became_peered":"2024-09-16T06:49:05.925903+0000","last_unstale":"2024-09-16T06:49:19.939632+0000","last_undegraded":"2024-09-16T06:49:19.939632+0000","last_fullsized":"2024-09-16T06:49:19.939632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.047883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016170553000000001,"stat_sum":{"num_bytes":245760,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":1347,"num_write":26,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,10,3,8,7,14],"acting":[13,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"20","length":"2"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"3.a","version":"91'188","reported_seq":336,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T06:49:18.943780+0000","last_change":"2024-09-16T06:49:18.944502+0000","last_active":"2024-09-16T06:49:18.943780+0000","last_peered":"2024-09-16T06:49:18.943780+0000","last_clean":"2024-09-16T06:49:18.943780+0000","last_became_active":"2024-09-16T06:48:20.885805+0000","last_became_peered":"2024-09-16T06:48:20.885805+0000","last_unstale":"2024-09-16T06:49:18.943780+0000","last_undegraded":"2024-09-16T06:49:18.943780+0000","last_fullsized":"2024-09-16T06:49:18.943780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:22.911175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010783612,"stat_sum":{"num_bytes":12435456,"num_objects":13,"num_object_clones":8,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":33,"num_read_kb":8509,"num_write":49,"num_write_kb":12976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,10,8,15,11],"acting":[3,7,6,10,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"3.9","version":"92'48","reported_seq":225,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.936941+0000","last_change":"2024-09-16T06:49:19.936941+0000","last_active":"2024-09-16T06:49:19.936941+0000","last_peered":"2024-09-16T06:49:19.936941+0000","last_clean":"2024-09-16T06:49:19.936941+0000","last_became_active":"2024-09-16T06:48:21.124872+0000","last_became_peered":"2024-09-16T06:48:21.124872+0000","last_unstale":"2024-09-16T06:49:19.936941+0000","last_undegraded":"2024-09-16T06:49:19.936941+0000","last_fullsized":"2024-09-16T06:49:19.936941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:43.972532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013681723,"stat_sum":{"num_bytes":8454144,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":18150,"num_write":22,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,10,4,15,9,1],"acting":[12,2,10,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"20","length":"2"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"3.8","version":"88'40","reported_seq":195,"reported_epoch":92,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:19.922944+0000","last_change":"2024-09-16T06:49:11.869149+0000","last_active":"2024-09-16T06:49:19.922944+0000","last_peered":"2024-09-16T06:49:19.922944+0000","last_clean":"2024-09-16T06:49:09.868266+0000","last_became_active":"2024-09-16T06:49:11.869149+0000","last_became_peered":"2024-09-16T06:49:11.869149+0000","last_unstale":"2024-09-16T06:49:19.922944+0000","last_undegraded":"2024-09-16T06:49:11.861397+0000","last_fullsized":"2024-09-16T06:49:11.860631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":7,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:23.160181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015152634999999999,"stat_sum":{"num_bytes":3801088,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":7,"num_read_kb":4,"num_write":33,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,2147483647,15,2],"acting":[6,1,0,4,2147483647,15,2],"avail_no_missing":["6(0)","0(2)","1(1)","2(6)","4(3)","15(5)"],"object_location_counts":[{"shards":"0(2),1(1),2(6),4(3),6(0),15(5)","objects":5}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"28'157","reported_seq":274,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.923557+0000","last_change":"2024-09-16T06:48:08.414696+0000","last_active":"2024-09-16T06:49:19.923557+0000","last_peered":"2024-09-16T06:49:19.923557+0000","last_clean":"2024-09-16T06:49:19.923557+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:49:19.923557+0000","last_undegraded":"2024-09-16T06:49:19.923557+0000","last_fullsized":"2024-09-16T06:49:19.923557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:48:06.616578+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T06:49:15.955067+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:49:15.955067+0000","last_peered":"2024-09-16T06:49:15.955067+0000","last_clean":"2024-09-16T06:49:15.955067+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:49:15.955067+0000","last_undegraded":"2024-09-16T06:49:15.955067+0000","last_fullsized":"2024-09-16T06:49:15.955067+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"89'146","reported_seq":295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T06:49:16.861584+0000","last_change":"2024-09-16T06:49:16.840477+0000","last_active":"2024-09-16T06:49:16.861584+0000","last_peered":"2024-09-16T06:49:16.861584+0000","last_clean":"2024-09-16T06:49:16.861584+0000","last_became_active":"2024-09-16T06:48:20.888283+0000","last_became_peered":"2024-09-16T06:48:20.888283+0000","last_unstale":"2024-09-16T06:49:16.861584+0000","last_undegraded":"2024-09-16T06:49:16.861584+0000","last_fullsized":"2024-09-16T06:49:16.861584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.236427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004584828,"stat_sum":{"num_bytes":9420800,"num_objects":12,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":28,"num_read_kb":6280,"num_write":120,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,10,11],"acting":[14,12,3,4,15,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"20","length":"1"},{"start":"29","length":"1"},{"start":"2c","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T06:49:17.829623+0000","last_change":"2024-09-16T06:48:11.767190+0000","last_active":"2024-09-16T06:49:17.829623+0000","last_peered":"2024-09-16T06:49:17.829623+0000","last_clean":"2024-09-16T06:49:17.829623+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:49:17.829623+0000","last_undegraded":"2024-09-16T06:49:17.829623+0000","last_fullsized":"2024-09-16T06:49:17.829623+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"92'63","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.951044+0000","last_change":"2024-09-16T06:49:19.952666+0000","last_active":"2024-09-16T06:49:19.951044+0000","last_peered":"2024-09-16T06:49:19.951044+0000","last_clean":"2024-09-16T06:49:19.951044+0000","last_became_active":"2024-09-16T06:49:05.936871+0000","last_became_peered":"2024-09-16T06:49:05.936871+0000","last_unstale":"2024-09-16T06:49:19.951044+0000","last_undegraded":"2024-09-16T06:49:19.951044+0000","last_fullsized":"2024-09-16T06:49:19.951044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:20.893875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027781587999999999,"stat_sum":{"num_bytes":5193728,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8309,"num_write":25,"num_write_kb":5664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5292032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"20","length":"2"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T06:49:15.955093+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:49:15.955093+0000","last_peered":"2024-09-16T06:49:15.955093+0000","last_clean":"2024-09-16T06:49:15.955093+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:49:15.955093+0000","last_undegraded":"2024-09-16T06:49:15.955093+0000","last_fullsized":"2024-09-16T06:49:15.955093+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.234200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"79'62","reported_seq":235,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.924081+0000","last_change":"2024-09-16T06:49:19.924081+0000","last_active":"2024-09-16T06:49:19.924081+0000","last_peered":"2024-09-16T06:49:19.924081+0000","last_clean":"2024-09-16T06:49:19.924081+0000","last_became_active":"2024-09-16T06:48:21.124782+0000","last_became_peered":"2024-09-16T06:48:21.124782+0000","last_unstale":"2024-09-16T06:49:19.924081+0000","last_undegraded":"2024-09-16T06:49:19.924081+0000","last_fullsized":"2024-09-16T06:49:19.924081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.918920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098583500000000001,"stat_sum":{"num_bytes":4866048,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":774,"num_write":21,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"20","length":"2"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T06:49:17.829958+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:49:17.829958+0000","last_peered":"2024-09-16T06:49:17.829958+0000","last_clean":"2024-09-16T06:49:17.829958+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:49:17.829958+0000","last_undegraded":"2024-09-16T06:49:17.829958+0000","last_fullsized":"2024-09-16T06:49:17.829958+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:31.486215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"87'126","reported_seq":298,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T06:49:16.821650+0000","last_change":"2024-09-16T06:49:16.822079+0000","last_active":"2024-09-16T06:49:16.821650+0000","last_peered":"2024-09-16T06:49:16.821650+0000","last_clean":"2024-09-16T06:49:16.821650+0000","last_became_active":"2024-09-16T06:48:21.121223+0000","last_became_peered":"2024-09-16T06:48:21.121223+0000","last_unstale":"2024-09-16T06:49:16.821650+0000","last_undegraded":"2024-09-16T06:49:16.821650+0000","last_fullsized":"2024-09-16T06:49:16.821650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.389211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084365199999999997,"stat_sum":{"num_bytes":12648448,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":52,"num_read_kb":18883,"num_write":86,"num_write_kb":13184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"20","length":"1"},{"start":"29","length":"1"},{"start":"2c","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.5","version":"82'12","reported_seq":211,"reported_epoch":89,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:16.845434+0000","last_change":"2024-09-16T06:49:11.868567+0000","last_active":"2024-09-16T06:49:16.845434+0000","last_peered":"2024-09-16T06:49:16.845434+0000","last_clean":"2024-09-16T06:49:09.867185+0000","last_became_active":"2024-09-16T06:49:11.868567+0000","last_became_peered":"2024-09-16T06:49:11.868567+0000","last_unstale":"2024-09-16T06:49:16.845434+0000","last_undegraded":"2024-09-16T06:49:11.859372+0000","last_fullsized":"2024-09-16T06:49:11.859033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:33.827554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017739370000000001,"stat_sum":{"num_bytes":2555904,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":17765,"num_write":7,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,2147483647,2,7,1,8],"acting":[15,13,2147483647,2,7,1,8],"avail_no_missing":["15(0)","1(5)","2(3)","7(4)","8(6)","13(1)"],"object_location_counts":[{"shards":"1(5),2(3),7(4),8(6),13(1),15(0)","objects":2}],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":77,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T06:49:15.891686+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:49:15.891686+0000","last_peered":"2024-09-16T06:49:15.891686+0000","last_clean":"2024-09-16T06:49:15.891686+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:49:15.891686+0000","last_undegraded":"2024-09-16T06:49:15.891686+0000","last_fullsized":"2024-09-16T06:49:15.891686+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:19.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"87'136","reported_seq":359,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T06:49:16.856912+0000","last_change":"2024-09-16T06:49:16.858470+0000","last_active":"2024-09-16T06:49:16.856912+0000","last_peered":"2024-09-16T06:49:16.856912+0000","last_clean":"2024-09-16T06:49:16.856912+0000","last_became_active":"2024-09-16T06:49:05.920150+0000","last_became_peered":"2024-09-16T06:49:05.920150+0000","last_unstale":"2024-09-16T06:49:16.856912+0000","last_undegraded":"2024-09-16T06:49:16.856912+0000","last_fullsized":"2024-09-16T06:49:16.856912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:38.017254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00161264,"stat_sum":{"num_bytes":6979584,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":41,"num_read_kb":30973,"num_write":52,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9486336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,2,6,10,13],"acting":[1,12,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"29","length":"1"},{"start":"2c","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":92,"state":"active+undersized","last_fresh":"2024-09-16T06:49:19.934576+0000","last_change":"2024-09-16T06:49:11.861382+0000","last_active":"2024-09-16T06:49:19.934576+0000","last_peered":"2024-09-16T06:49:19.934576+0000","last_clean":"2024-09-16T06:49:09.866035+0000","last_became_active":"2024-09-16T06:49:11.861382+0000","last_became_peered":"2024-09-16T06:49:11.861382+0000","last_unstale":"2024-09-16T06:49:19.934576+0000","last_undegraded":"2024-09-16T06:49:19.934576+0000","last_fullsized":"2024-09-16T06:49:11.860463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:32.168820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8],"acting":[8],"avail_no_missing":["8"],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"92'132","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.973122+0000","last_change":"2024-09-16T06:49:19.974578+0000","last_active":"2024-09-16T06:49:19.973122+0000","last_peered":"2024-09-16T06:49:19.973122+0000","last_clean":"2024-09-16T06:49:19.973122+0000","last_became_active":"2024-09-16T06:49:05.952263+0000","last_became_peered":"2024-09-16T06:49:05.952263+0000","last_unstale":"2024-09-16T06:49:19.973122+0000","last_undegraded":"2024-09-16T06:49:19.973122+0000","last_fullsized":"2024-09-16T06:49:19.973122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:24.703234+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.049312466999999999,"stat_sum":{"num_bytes":17481728,"num_objects":11,"num_object_clones":7,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":37,"num_read_kb":20656,"num_write":75,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":34914304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"20","length":"2"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T06:49:15.898793+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:49:15.898793+0000","last_peered":"2024-09-16T06:49:15.898793+0000","last_clean":"2024-09-16T06:49:15.898793+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:49:15.898793+0000","last_undegraded":"2024-09-16T06:49:15.898793+0000","last_fullsized":"2024-09-16T06:49:15.898793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"92'24","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T06:49:19.948845+0000","last_change":"2024-09-16T06:49:19.950493+0000","last_active":"2024-09-16T06:49:19.948845+0000","last_peered":"2024-09-16T06:49:19.948845+0000","last_clean":"2024-09-16T06:49:19.948845+0000","last_became_active":"2024-09-16T06:49:05.928404+0000","last_became_peered":"2024-09-16T06:49:05.928404+0000","last_unstale":"2024-09-16T06:49:19.948845+0000","last_undegraded":"2024-09-16T06:49:19.948845+0000","last_fullsized":"2024-09-16T06:49:19.948845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:33.747452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.025227469999999998,"stat_sum":{"num_bytes":2834432,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":10278,"num_write":7,"num_write_kb":2768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2899968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,6,2,1,7,0],"acting":[13,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"20","length":"2"},{"start":"29","length":"4"},{"start":"35","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T06:49:15.897986+0000","last_change":"2024-09-16T06:49:12.584897+0000","last_active":"2024-09-16T06:49:15.897986+0000","last_peered":"2024-09-16T06:49:15.897986+0000","last_clean":"2024-09-16T06:49:15.897986+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:49:15.897986+0000","last_undegraded":"2024-09-16T06:49:15.897986+0000","last_fullsized":"2024-09-16T06:49:15.897986+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:12.584669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:12.584669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:13.936208+0000","scrub_duration":0.0037478810000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":123453440,"num_objects":107,"num_object_clones":59,"num_object_copies":749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":8,"num_read":379,"num_read_kb":178353,"num_write":677,"num_write_kb":130144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":100876288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":209920000,"data_stored":332673024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1365,"ondisk_log_size":1365,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":15,"acting":15,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247824,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44056,"kb_used_data":12936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327784,"statfs":{"total":96636764160,"available":96591650816,"internally_reserved":0,"allocated":13246464,"data_stored":18870358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":45.249000000000002,"5min":45.249000000000002,"15min":45.249000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":780.85000000000002,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.98799999999999999},{"interface":"front","average":{"1min":9.7080000000000002,"5min":9.7080000000000002,"15min":9.7080000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":164.398,"5min":164.398,"15min":164.398},"last":0.72299999999999998}]},{"osd":1,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.90800000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.145}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.077},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":13.01,"5min":13.01,"15min":13.01},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":225.697,"5min":225.697,"15min":225.697},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.1699999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":1.2909999999999999},{"interface":"front","average":{"1min":12.073,"5min":12.073,"15min":12.073},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":197.559,"5min":197.559,"15min":197.559},"last":1.268}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.367},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.3480000000000001}]}]},{"osd":14,"up_from":23,"seq":98784247826,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43900,"kb_used_data":12776,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327940,"statfs":{"total":96636764160,"available":96591810560,"internally_reserved":0,"allocated":13082624,"data_stored":21244225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":27.170999999999999,"5min":27.170999999999999,"15min":27.170999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":525.66800000000001,"5min":525.66800000000001,"15min":525.66800000000001},"last":1.026},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.1950000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.091}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.91900000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.452}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":360.42200000000003,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":3.7709999999999999,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.399}]}]},{"osd":11,"up_from":23,"seq":98784247825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49980,"kb_used_data":18856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94321860,"statfs":{"total":96636764160,"available":96585584640,"internally_reserved":0,"allocated":19308544,"data_stored":31938881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":50.844000000000001,"5min":50.844000000000001,"15min":50.844000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":949.71199999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.135}]},{"osd":1,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":48.140999999999998,"5min":48.140999999999998,"15min":48.140999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":894.49900000000002,"5min":894.49900000000002,"15min":894.49900000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":781.33199999999999},{"interface":"front","average":{"1min":4.8739999999999997,"5min":4.8739999999999997,"15min":4.8739999999999997},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":71.503,"5min":71.503,"15min":71.503},"last":1.1100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":4,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.024},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.7400000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":722.72199999999998}]},{"osd":10,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6469999999999998,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.048}]},{"osd":12,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.0820000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":14,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.94299999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":3,"up_from":21,"seq":90194313234,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38476,"kb_used_data":11452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333364,"statfs":{"total":96636764160,"available":96597364736,"internally_reserved":0,"allocated":11726848,"data_stored":15706680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,4,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.80700000000000005}]},{"osd":4,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.64200000000000002},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":9,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.008}]},{"osd":10,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":42.863,"5min":42.863,"15min":42.863},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":502.57400000000001,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":41.012,"5min":41.012,"15min":41.012},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":502.62200000000001,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.133}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.115}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]}]},{"osd":13,"up_from":21,"seq":90194313234,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57016,"kb_used_data":25900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94314824,"statfs":{"total":96636764160,"available":96578379776,"internally_reserved":0,"allocated":26521600,"data_stored":32314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1470000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":4.2089999999999996},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":8.6449999999999996,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":4.3129999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":4.5419999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":4.2619999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":28.696999999999999,"5min":28.696999999999999,"15min":28.696999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":385.83800000000002,"5min":385.83800000000002,"15min":385.83800000000002},"last":4.6600000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":4.5919999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5119999999999996}]},{"osd":8,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":4.6900000000000004},{"interface":"front","average":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":96.432000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":4.5750000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7160000000000002}]},{"osd":11,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":10.728,"5min":10.728,"15min":10.728},"last":4.6349999999999998},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":3.8069999999999999,"5min":3.8069999999999999,"15min":3.8069999999999999},"last":4.7699999999999996}]},{"osd":12,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":4.8949999999999996},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":4.8630000000000004}]},{"osd":14,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":4.9870000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2,"5min":2,"15min":2},"last":4.9260000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":5880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":6021120,"data_stored":7533174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":39.485999999999997,"5min":39.485999999999997,"15min":39.485999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":734.39200000000005,"5min":734.39200000000005,"15min":734.39200000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":2.3679999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":2.3250000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.625}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.873},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.2470000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":2.399},{"interface":"front","average":{"1min":32.578000000000003,"5min":32.578000000000003,"15min":32.578000000000003},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":596.70600000000002,"5min":596.70600000000002,"15min":596.70600000000002},"last":1.2709999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.123},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.2150000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":2.427},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.7149999999999999,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":2.5070000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":2.4870000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":3.1339999999999999}]}]},{"osd":4,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46824,"kb_used_data":15700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325016,"statfs":{"total":96636764160,"available":96588816384,"internally_reserved":0,"allocated":16076800,"data_stored":26722984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":3,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":14.775,"5min":14.775,"15min":14.775},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":265.48099999999999,"5min":265.48099999999999,"15min":265.48099999999999},"last":0.745},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.1539999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.133},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":1.1080000000000001}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":31.298999999999999,"5min":31.298999999999999,"15min":31.298999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":576.29200000000003,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.097},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.2649999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.99,"15min":1.99},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":20.283999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":16.338999999999999,"5min":16.338999999999999,"15min":16.338999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":306.85599999999999,"5min":306.85599999999999,"15min":306.85599999999999},"last":0.89400000000000002}]},{"osd":10,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.2949999999999999},{"interface":"front","average":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":21.588999999999999,"5min":21.588999999999999,"15min":21.588999999999999},"last":1.1799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.341},{"interface":"front","average":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":20.437999999999999,"5min":20.437999999999999,"15min":20.437999999999999},"last":1.252}]}]},{"osd":5,"up_from":22,"seq":94489280527,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34536,"kb_used_data":7508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337304,"statfs":{"total":96636764160,"available":96601399296,"internally_reserved":0,"allocated":7688192,"data_stored":10847401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,8,10,11,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1579999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]}]},{"osd":6,"up_from":22,"seq":94489280530,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45676,"kb_used_data":14560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94326164,"statfs":{"total":96636764160,"available":96589991936,"internally_reserved":0,"allocated":14909440,"data_stored":22131912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,7,8,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.006}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.1319999999999999}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.365},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.113}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.751},{"interface":"front","average":{"1min":13.279,"5min":13.279,"15min":13.279},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":231.90700000000001,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.0820000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.194},{"interface":"front","average":{"1min":18.225000000000001,"5min":18.225000000000001,"15min":18.225000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":323.55500000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.0600000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.94599999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4059999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.446},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.1759999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280532,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":10940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":11202560,"data_stored":14957249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,6,8,9,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.91900000000000004},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":0.82099999999999995}]},{"osd":1,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":1.216},{"interface":"front","average":{"1min":29.103999999999999,"5min":29.103999999999999,"15min":29.103999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":527.10299999999995,"5min":527.10299999999995,"15min":527.10299999999995},"last":1.173}]},{"osd":2,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":6.1749999999999998,"5min":6.1749999999999998,"15min":6.1749999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":136.72399999999999,"5min":136.72399999999999,"15min":136.72399999999999},"last":1.143},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":20.370999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":508.52100000000002,"5min":508.52100000000002,"15min":508.52100000000002},"last":1.018},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.3200000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.274},{"interface":"front","average":{"1min":30.073,"5min":30.073,"15min":30.073},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":415.30099999999999,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.99199999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":37.726999999999997,"5min":37.726999999999997,"15min":37.726999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":502.55200000000002,"5min":502.55200000000002,"15min":502.55200000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":20.564,"5min":20.564,"15min":20.564},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":502.39600000000002,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.4179999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.385},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.7250000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":21.131,"5min":21.131,"15min":21.131},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":514.07799999999997,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.4950000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":10212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":10457088,"data_stored":17384104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":10.706,"5min":10.706,"15min":10.706},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":177.92400000000001,"5min":177.92400000000001,"15min":177.92400000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.99099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.70599999999999996}]},{"osd":6,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":27.687000000000001,"5min":27.687000000000001,"15min":27.687000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":449.88,"5min":449.88,"15min":449.88},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.876}]},{"osd":9,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.014},{"interface":"front","average":{"1min":1.903,"5min":1.903,"15min":1.903},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":12.366,"5min":12.366,"15min":12.366},"last":1.091}]},{"osd":10,"last update":"Mon Sep 16 06:49:14 2024","interfaces":[{"interface":"back","average":{"1min":56.750999999999998,"5min":56.750999999999998,"15min":56.750999999999998},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":998.42399999999998,"5min":998.42399999999998,"15min":998.42399999999998},"last":1.24},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.2669999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.5869999999999997,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.29},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.1930000000000001}]}]},{"osd":12,"up_from":22,"seq":94489280531,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48804,"kb_used_data":17688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323036,"statfs":{"total":96636764160,"available":96586788864,"internally_reserved":0,"allocated":18112512,"data_stored":27142849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.411}]},{"osd":3,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.4119999999999999,"5min":3.4119999999999999,"15min":3.4119999999999999},"last":1.157},{"interface":"front","average":{"1min":38.109000000000002,"5min":38.109000000000002,"15min":38.109000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":708.49699999999996,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.86699999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":33.972000000000001,"5min":33.972000000000001,"15min":33.972000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":630.75900000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.72099999999999997},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.288},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.083}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.1100000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.3330000000000002,"5min":4.3330000000000002,"15min":4.3330000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":1.2350000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.995},{"interface":"front","average":{"1min":50.134999999999998,"5min":50.134999999999998,"15min":50.134999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":980.38300000000004,"5min":980.38300000000004,"15min":980.38300000000004},"last":1.2669999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":1.381}]},{"osd":14,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.2090000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.4370000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247827,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50732,"kb_used_data":19612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94321108,"statfs":{"total":96636764160,"available":96584814592,"internally_reserved":0,"allocated":20082688,"data_stored":34054230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":12.209,"5min":12.209,"15min":12.209},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":214.93299999999999,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.872},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0449999999999999}]},{"osd":2,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.997},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":12.398,"5min":12.398,"15min":12.398},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":216.958,"5min":216.958,"15min":216.958},"last":1.258},{"interface":"front","average":{"1min":35.026000000000003,"5min":35.026000000000003,"15min":35.026000000000003},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":645.88099999999997,"5min":645.88099999999997,"15min":645.88099999999997},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.95299999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":28.591000000000001,"5min":28.591000000000001,"15min":28.591000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":493.76100000000002,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.75900000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":11,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":37.526000000000003,"5min":37.526000000000003,"15min":37.526000000000003},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":687.10799999999995,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.083},{"interface":"front","average":{"1min":26.273,"5min":26.273,"15min":26.273},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":475.75700000000001,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.1890000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.024}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]}]},{"osd":2,"up_from":23,"seq":98784247825,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44640,"kb_used_data":13520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327200,"statfs":{"total":96636764160,"available":96591052800,"internally_reserved":0,"allocated":13844480,"data_stored":19361878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.77500000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.627},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.79100000000000004}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":19.053000000000001,"5min":19.053000000000001,"15min":19.053000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":361.36799999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":35.137999999999998,"5min":35.137999999999998,"15min":35.137999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":646.98800000000006,"5min":646.98800000000006,"15min":646.98800000000006},"last":0.71699999999999997},{"interface":"front","average":{"1min":28.411999999999999,"5min":28.411999999999999,"15min":28.411999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":516.49000000000001,"5min":516.49000000000001,"15min":516.49000000000001},"last":0.755}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.8239999999999998,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.95699999999999996}]},{"osd":10,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.89500000000000002}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":0.92400000000000004}]}]},{"osd":9,"up_from":23,"seq":98784247824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41716,"kb_used_data":10596,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94330124,"statfs":{"total":96636764160,"available":96594046976,"internally_reserved":0,"allocated":10850304,"data_stored":15618153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":50.427999999999997,"5min":50.427999999999997,"15min":50.427999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":941.91899999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.753},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.0609999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.038}]},{"osd":6,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.68500000000000005}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":104.761,"5min":104.761,"15min":104.761},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":20.274000000000001,"5min":20.274000000000001,"15min":20.274000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":363.67399999999998,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.325},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.1459999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.169},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.375}]},{"osd":12,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":1.1870000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.284},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.0129999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.204},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.4059999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.224},{"interface":"front","average":{"1min":49.017000000000003,"5min":49.017000000000003,"15min":49.017000000000003},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":750.83299999999997,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.27}]}]},{"osd":10,"up_from":23,"seq":98784247825,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48108,"kb_used_data":16984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323732,"statfs":{"total":96636764160,"available":96587501568,"internally_reserved":0,"allocated":17391616,"data_stored":28473665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":31848898},"hb_peers":[1,2,3,4,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.85799999999999998},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.071},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.046}]},{"osd":9,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.98699999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.135}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":6270976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18804736,"data_stored":33320960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12566528,"data_stored":18628608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10395648,"data_stored":14942208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14733312,"data_stored":25948160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7049216,"data_stored":10805248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13033472,"data_stored":20828160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9842688,"data_stored":14172160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":9113600,"data_stored":16609280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9568256,"data_stored":14884864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16343040,"data_stored":27975680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":31195136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":16752640,"data_stored":26357760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":25178112,"data_stored":31539200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":11788288,"data_stored":20500480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":12525568,"data_stored":18694144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:49:22.904 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [9, 10, 8, 0, 1, 2, 12, 4, 11, 6, 15, 14, 7, 5] 2024-09-16T06:49:22.904 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.8', '9', '10', '8', '0', '1', '2', '12', '4', '11', '6', '15', '14', '7', '5'] 2024-09-16T06:49:22.905 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.8 9 10 8 0 1 2 12 4 11 6 15 14 7 5 2024-09-16T06:49:22.912 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:22.912 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:22.934 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:22.934 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:22.934 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:22.936 INFO:tasks.rados.rados.0.smithi117.stdout:327: done (3 left) 2024-09-16T06:49:22.936 INFO:tasks.rados.rados.0.smithi117.stdout:328: done (2 left) 2024-09-16T06:49:22.936 INFO:tasks.rados.rados.0.smithi117.stdout:329: done (1 left) 2024-09-16T06:49:22.936 INFO:tasks.rados.rados.0.smithi117.stdout:330: done (0 left) 2024-09-16T06:49:22.936 INFO:tasks.rados.rados.0.smithi117.stdout:331: snap_remove snap 13 2024-09-16T06:49:22.943 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:49:22.952 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:22.952 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:49:22.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:49:22.952 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:49:22.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.952 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 34/32 ECSubWrite(tid=1, reqid=client.4511.0:2, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 60.051567018, 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064369220000000003, 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899648+0000", 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899647+0000", 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899689+0000", 2024-09-16T06:49:22.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2429000000000003e-05 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899768+0000", 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9277999999999995e-05 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.899797+0000", 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8473999999999999e-05 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903489+0000", 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036918680000000001 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903519+0000", 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0283000000000001e-05 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906024+0000", 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025045430000000001 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906085+0000", 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1637000000000006e-05 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 61/32 ECSubWrite(tid=57, reqid=osd.12.0:57, at_version=61'40, trim_to=0'0, roll_forward_to=59'36))", 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.128354612999999, 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068954469999999999, 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822861+0000", 2024-09-16T06:49:22.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822859+0000", 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.822915+0000", 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6230999999999997e-05 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823009+0000", 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4176000000000001e-05 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823032+0000", 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3005000000000001e-05 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823531+0000", 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000498326 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.823611+0000", 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0495999999999996e-05 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829687+0000", 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060763509999999998 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:48.829756+0000", 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8765999999999999e-05 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.962 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.118079475999998, 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069218320000000002, 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:49:22.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833136+0000", 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833135+0000", 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833193+0000", 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8183999999999997e-05 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833314+0000", 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012045900000000001 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833341+0000", 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6954999999999999e-05 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833966+0000", 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00062526400000000005 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.833994+0000", 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7942000000000001e-05 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.840004+0000", 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060096969999999996 2024-09-16T06:49:22.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.840058+0000", 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3931000000000003e-05 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.967 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 71/54 ECSubWrite(tid=154, reqid=osd.3.0:149, at_version=0'0, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.075624608999998, 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076751120000000004, 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875591+0000", 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875589+0000", 2024-09-16T06:49:22.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875617+0000", 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8209999999999999e-05 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875704+0000", 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7047999999999997e-05 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875725+0000", 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0837000000000001e-05 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875784+0000", 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9077999999999999e-05 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.875794+0000", 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5319999999999995e-06 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883218+0000", 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0074248020000000003 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883266+0000", 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7339000000000002e-05 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 71/37 ECSubWrite(tid=129, reqid=osd.1.0:129, at_version=71'79, trim_to=0'0, roll_forward_to=71'77))", 2024-09-16T06:49:22.972 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.075108978999999, 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071128010000000002, 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876106+0000", 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876105+0000", 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876130+0000", 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.497e-05 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876185+0000", 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4926999999999998e-05 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876194+0000", 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7669999999999992e-06 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876307+0000", 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113189 2024-09-16T06:49:22.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.876326+0000", 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8553999999999999e-05 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883200+0000", 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006874092 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:58.883219+0000", 2024-09-16T06:49:22.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9286000000000001e-05 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 72/37 ECSubWrite(tid=214, reqid=client.4511.0:614, at_version=72'94, trim_to=0'0, roll_forward_to=72'92))", 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.657014+0000", 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.294201511000001, 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0097847630000000001, 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657014+0000", 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657014+0000", 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657012+0000", 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:22.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657200+0000", 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018793999999999999 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657310+0000", 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011035900000000001 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657333+0000", 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2639000000000001e-05 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657450+0000", 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116953 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657466+0000", 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5875999999999998e-05 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666720+0000", 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.009254623 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666799+0000", 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.805e-05 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 72/37 ECSubWrite(tid=215, reqid=client.4511.0:615, at_version=72'95, trim_to=0'0, roll_forward_to=72'92))", 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.657860+0000", 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.293354809, 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089789369999999993, 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657860+0000", 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657860+0000", 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657859+0000", 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:22.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.657921+0000", 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1810999999999995e-05 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658008+0000", 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7837000000000006e-05 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658029+0000", 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658488+0000", 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000459087 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.658517+0000", 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9088999999999999e-05 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666817+0000", 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0082997209999999995 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.666839+0000", 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2643e-05 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.9s5 73/32 ECSubWrite(tid=79, reqid=osd.12.0:79, at_version=73'17, trim_to=0'0, roll_forward_to=72'15))", 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.099801996, 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0095406529999999996, 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.987 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851413+0000", 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851471+0000", 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7877999999999997e-05 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853188+0000", 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017171039999999999 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853213+0000", 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5352000000000001e-05 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858138+0000", 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049245390000000003 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858175+0000", 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7518999999999998e-05 2024-09-16T06:49:22.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.860884+0000", 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002708907 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.860954+0000", 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9877999999999997e-05 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 73/37 ECSubWrite(tid=219, reqid=osd.11.0:219, at_version=73'97, trim_to=0'0, roll_forward_to=72'95))", 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.851564+0000", 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.099651345000002, 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0097268860000000006, 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851564+0000", 2024-09-16T06:49:22.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851564+0000", 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851563+0000", 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851666+0000", 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103199 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851771+0000", 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000104616 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851794+0000", 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2923000000000001e-05 2024-09-16T06:49:22.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.851983+0000", 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018950999999999999 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.852011+0000", 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7705e-05 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.861252+0000", 2024-09-16T06:49:22.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092412039999999994 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.861291+0000", 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8439000000000002e-05 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.996 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.0s6 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.853271+0000", 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.097944471000002, 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010549976000000001, 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853271+0000", 2024-09-16T06:49:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853271+0000", 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853269+0000", 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:22.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853322+0000", 2024-09-16T06:49:22.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3031000000000001e-05 2024-09-16T06:49:22.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:22.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:22.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:22.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853536+0000", 2024-09-16T06:49:22.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021384900000000001 2024-09-16T06:49:23.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853561+0000", 2024-09-16T06:49:23.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4998999999999999e-05 2024-09-16T06:49:23.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858933+0000", 2024-09-16T06:49:23.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053721029999999996 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858973+0000", 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9894999999999997e-05 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.863755+0000", 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004782204 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.863821+0000", 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5314999999999994e-05 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 73/37 ECSubWrite(tid=152, reqid=osd.1.0:152, at_version=73'89, trim_to=0'0, roll_forward_to=72'87))", 2024-09-16T06:49:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.853363+0000", 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.097852514, 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0091925289999999996, 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853363+0000", 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853363+0000", 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853362+0000", 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853407+0000", 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4335000000000002e-05 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853621+0000", 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000214389 2024-09-16T06:49:23.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.853638+0000", 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7173e-05 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858504+0000", 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048660470000000001 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.858533+0000", 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9048999999999999e-05 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862470+0000", 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039370230000000004 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862555+0000", 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4937000000000003e-05 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.009 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 73/54 ECSubWrite(tid=182, reqid=osd.3.0:182, at_version=73'67, trim_to=0'0, roll_forward_to=72'65))", 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.096974190000001, 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0083643499999999996, 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854241+0000", 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854296+0000", 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5146999999999999e-05 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854438+0000", 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000142115 2024-09-16T06:49:23.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.854460+0000", 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2453e-05 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.859010+0000", 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004550119 2024-09-16T06:49:23.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.859040+0000", 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9606000000000001e-05 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862528+0000", 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034876859999999998 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:00.862605+0000", 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7733999999999998e-05 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.ds4 82/77 ECSubWrite(tid=222, reqid=osd.3.0:219, at_version=0'0, trim_to=0'0, roll_forward_to=82'77))", 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.865487+0000", 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.085727817, 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0082477100000000001, 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.015 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865487+0000", 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865487+0000", 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865485+0000", 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865515+0000", 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0229000000000001e-05 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865772+0000", 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025714599999999998 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.865796+0000", 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3938999999999999e-05 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.866003+0000", 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020710100000000001 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.866030+0000", 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6216000000000001e-05 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873679+0000", 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076493029999999997 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873735+0000", 2024-09-16T06:49:23.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6276000000000001e-05 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 82/77 ECSubWrite(tid=186, reqid=osd.1.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=82'114))", 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.867995+0000", 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.083220372, 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0090620000000000006, 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.867995+0000", 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.867995+0000", 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.867993+0000", 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:23.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868021+0000", 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8405999999999999e-05 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868127+0000", 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010522 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868149+0000", 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2543000000000001e-05 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868501+0000", 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035184000000000002 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.868558+0000", 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6872e-05 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877001+0000", 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084430970000000001 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877057+0000", 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5886e-05 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.024 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 82/77 ECSubWrite(tid=187, reqid=osd.1.0:187, at_version=82'116, trim_to=0'0, roll_forward_to=82'114))", 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.869201+0000", 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.082014536999999, 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079066459999999998, 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869201+0000", 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869201+0000", 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869199+0000", 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869252+0000", 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3041999999999998e-05 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869351+0000", 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9164000000000001e-05 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.869372+0000", 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1223e-05 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870145+0000", 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077266399999999995 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870174+0000", 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9218999999999999e-05 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877074+0000", 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068994089999999996 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.877107+0000", 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3809000000000001e-05 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 82/81 ECSubWrite(tid=255, reqid=osd.11.0:255, at_version=82'106, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.870272+0000", 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.080942847999999, 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0086404859999999993, 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.029 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870272+0000", 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870272+0000", 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870271+0000", 2024-09-16T06:49:23.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870333+0000", 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1748999999999994e-05 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870413+0000", 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0340999999999995e-05 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.870433+0000", 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0029000000000001e-05 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871092+0000", 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00065900799999999999 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871113+0000", 2024-09-16T06:49:23.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0968999999999999e-05 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878880+0000", 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007766935 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878913+0000", 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3039e-05 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 82/77 ECSubWrite(tid=188, reqid=osd.1.0:188, at_version=82'118, trim_to=0'0, roll_forward_to=82'114))", 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.871975+0000", 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.079240286999999, 2024-09-16T06:49:23.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068971400000000004, 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871975+0000", 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871975+0000", 2024-09-16T06:49:23.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.871946+0000", 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999709 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872054+0000", 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108447 2024-09-16T06:49:23.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872176+0000", 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012151599999999999 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872199+0000", 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3479999999999999e-05 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872377+0000", 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017830799999999999 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.872402+0000", 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4646000000000001e-05 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878805+0000", 2024-09-16T06:49:23.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064029430000000003 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.878872+0000", 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7088999999999994e-05 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.cs6 82/81 ECSubWrite(tid=256, reqid=osd.11.0:256, at_version=82'108, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.873698+0000", 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.077517759999999, 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079502619999999996, 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873698+0000", 2024-09-16T06:49:23.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873698+0000", 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873696+0000", 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873751+0000", 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4925e-05 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873867+0000", 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115967 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.873889+0000", 2024-09-16T06:49:23.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2463000000000002e-05 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.875076+0000", 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011874089999999999 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.875100+0000", 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3142999999999998e-05 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.881595+0000", 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064951879999999998 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.881648+0000", 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3026999999999999e-05 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.4s2 87/77 ECSubWrite(tid=200, reqid=osd.1.0:200, at_version=87'132, trim_to=0'0, roll_forward_to=85'128))", 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:14.904950+0000", 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.0462649469999992, 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073431149999999999, 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.904950+0000", 2024-09-16T06:49:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.904950+0000", 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.904949+0000", 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.904997+0000", 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8619000000000003e-05 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.905094+0000", 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6174000000000004e-05 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.905117+0000", 2024-09-16T06:49:23.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3445999999999999e-05 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.905611+0000", 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049440400000000005 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.905635+0000", 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3183000000000001e-05 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.912228+0000", 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0065934269999999998 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:14.912293+0000", 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5432000000000006e-05 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWrite(3.9s5 90/32 ECSubWrite(tid=115, reqid=osd.12.0:115, at_version=90'38, trim_to=0'0, roll_forward_to=87'36))", 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:17.834820+0000", 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.116395507, 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076495740000000001, 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.834820+0000", 2024-09-16T06:49:23.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.834820+0000", 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.834819+0000", 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.834866+0000", 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6576000000000003e-05 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.834968+0000", 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010177400000000001 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.835019+0000", 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1827999999999999e-05 2024-09-16T06:49:23.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.835150+0000", 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000130827 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.835176+0000", 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5269000000000002e-05 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:23.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.842399+0000", 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0072234580000000003 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:17.842469+0000", 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0364999999999995e-05 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:23.054 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:23.056 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:23.056 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:23.057 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:23.057 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:23.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:49:23.058 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:23.058 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:23.059 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:23.059 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:23.059 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:23.059 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:23.060 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:23.054+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:23.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:23.079 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:23.113 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:23.149 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:23.150 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:23.157 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:23.160+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:23.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:23.188 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:23.189 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:23.189 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:23.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:49:23.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:23.220+0000 7f4e8bfff640 0 -- 172.21.15.100:0/1989188102 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f4e8c21f030 msgr2=0x7f4e8c221420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:23.223 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:23.227 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:23.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:23.271 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:23.276+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:23.324 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:23.324 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:23.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:49:23.359 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:23.358+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:23.456 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:23.457 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:23.460 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:23.464+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:23.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:49:23.561 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:23.561+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:23.601 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:23.601 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:23.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:49:23.661 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:23.661+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:23.737 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:23.737 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:23.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:49:23.762 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:23.762+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:23.863 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:23.868+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:23.873 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:23.873 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:23.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:49:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:49:23.952 INFO:teuthology.orchestra.run.smithi100.stderr:set 3.8 pg_upmap_items mapping to [9->10,8->0,1->2,12->4,11->6,15->14,7->5] 2024-09-16T06:49:23.954 INFO:tasks.rados.rados.0.smithi117.stdout:331: done (0 left) 2024-09-16T06:49:23.954 INFO:tasks.rados.rados.0.smithi117.stdout:332: rmattr oid 17 current snap is 35 2024-09-16T06:49:23.954 INFO:tasks.rados.rados.0.smithi117.stdout:332: done (0 left) 2024-09-16T06:49:23.954 INFO:tasks.rados.rados.0.smithi117.stdout:333: delete oid 48 current snap is 35 2024-09-16T06:49:23.964 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:23.958+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:23.977 INFO:tasks.rados.rados.0.smithi117.stdout:333: done (0 left) 2024-09-16T06:49:23.977 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 27 current snap is 35 2024-09-16T06:49:23.977 INFO:tasks.rados.rados.0.smithi117.stdout:334: seq_num 138 ranges {2228224=770048,2998272=540672,3538944=753664,4292608=262144} 2024-09-16T06:49:24.019 INFO:tasks.rados.rados.0.smithi117.stdout:334: writing smithi11731613-27 from 2228224 to 2998272 tid 1 2024-09-16T06:49:24.026 INFO:tasks.rados.rados.0.smithi117.stdout:334: writing smithi11731613-27 from 2998272 to 3538944 tid 2 2024-09-16T06:49:24.031 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.031 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.035 INFO:tasks.rados.rados.0.smithi117.stdout:334: writing smithi11731613-27 from 3538944 to 4292608 tid 3 2024-09-16T06:49:24.038 INFO:tasks.rados.rados.0.smithi117.stdout:334: writing smithi11731613-27 from 4292608 to 4554752 tid 4 2024-09-16T06:49:24.038 INFO:tasks.rados.rados.0.smithi117.stdout:335: read oid 44 snap -1 2024-09-16T06:49:24.038 INFO:tasks.rados.rados.0.smithi117.stdout:335: expect (ObjNum 135 snap 33 seq_num 135) 2024-09-16T06:49:24.038 INFO:tasks.rados.rados.0.smithi117.stdout:336: snap_create 2024-09-16T06:49:24.038 INFO:tasks.rados.rados.0.smithi117.stdout:334: finishing write tid 1 to smithi11731613-27 2024-09-16T06:49:24.042 INFO:tasks.rados.rados.0.smithi117.stdout:334: finishing write tid 2 to smithi11731613-27 2024-09-16T06:49:24.042 INFO:tasks.rados.rados.0.smithi117.stdout:334: finishing write tid 3 to smithi11731613-27 2024-09-16T06:49:24.046 INFO:tasks.rados.rados.0.smithi117.stdout:334: finishing write tid 4 to smithi11731613-27 2024-09-16T06:49:24.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:49:24.064 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:24.061+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:24.165 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:24.162+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:24.166 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.166 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.179 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:49:24.265 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:24.262+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:24.299 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.299 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:49:24.367 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:24.366+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:24.467 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:24.462+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:24.482 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.483 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:49:24.568 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:24.566+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:24.614 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.614 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:49:24.669 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:24.669+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:24.724 INFO:tasks.rados.rados.0.smithi117.stdout:334: finishing write tid 5 to smithi11731613-27 2024-09-16T06:49:24.726 INFO:tasks.rados.rados.0.smithi117.stdout:334: finishing write tid 6 to smithi11731613-27 2024-09-16T06:49:24.726 INFO:tasks.rados.rados.0.smithi117.stdout:334: oid 27 updating version 0 to 176 2024-09-16T06:49:24.727 INFO:tasks.rados.rados.0.smithi117.stdout:334: oid 27 version 176 is already newer than 175 2024-09-16T06:49:24.727 INFO:tasks.rados.rados.0.smithi117.stdout:334: oid 27 version 176 is already newer than 173 2024-09-16T06:49:24.727 INFO:tasks.rados.rados.0.smithi117.stdout:334: oid 27 version 176 is already newer than 174 2024-09-16T06:49:24.727 INFO:tasks.rados.rados.0.smithi117.stdout:334: oid 27 version 176 is already newer than 172 2024-09-16T06:49:24.727 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 27 v 176 (ObjNum 138 snap 35 seq_num 138) dirty exists 2024-09-16T06:49:24.727 INFO:tasks.rados.rados.0.smithi117.stdout:334: left oid 27 (ObjNum 138 snap 35 seq_num 138) 2024-09-16T06:49:24.751 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.752 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.766 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:49:24.769 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:24.772+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:24.809 INFO:tasks.rados.rados.0.smithi117.stdout:335: expect (ObjNum 135 snap 33 seq_num 135) 2024-09-16T06:49:24.870 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:24.869+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:24.884 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:24.884 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:24.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:49:24.971 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:24.969+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:25.019 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:25.020 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:25.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:49:25.071 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:25.076+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:25.157 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:25.157 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:25.171 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:49:25.172 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:25.170+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:25.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:25.271 INFO:tasks.rados.rados.0.smithi117.stdout:334: done (2 left) 2024-09-16T06:49:25.271 INFO:tasks.rados.rados.0.smithi117.stdout:335: done (1 left) 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:336: done (0 left) 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:337: read oid 11 snap -1 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:337: expect (ObjNum 121 snap 28 seq_num 121) 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:338: copy_from oid 25 from oid 30 current snap is 36 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:339: read oid 8 snap -1 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:339: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:49:25.272 INFO:tasks.rados.rados.0.smithi117.stdout:340: copy_from oid 40 from oid 42 current snap is 36 2024-09-16T06:49:25.273 INFO:tasks.rados.rados.0.smithi117.stdout:341: snap_remove snap 29 2024-09-16T06:49:25.273 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:25.270+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:25.301 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:25.301 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:25.315 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:49:25.373 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:25.370+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:25.396 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:25.396 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:25.421 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:25.421 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:25.436 INFO:teuthology.orchestra.run.smithi117.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:25.436 INFO:teuthology.orchestra.run.smithi117.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:25.446 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:25.446 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:25.452 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:49:25.474 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:25.474 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:25.497 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:25.498+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:25.499 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:25.500 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:25.500 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:25.528 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:25.529 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:25.558 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:25.558 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:74 3.as0 3:5d679340:::smithi11731613-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.607279820999999, 2024-09-16T06:49:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026692272999999999, 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 74 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962276+0000", 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962275+0000", 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962280+0000", 2024-09-16T06:49:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1300000000000003e-06 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962286+0000", 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4529999999999999e-06 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962292+0000", 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5849999999999999e-06 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964248+0000", 2024-09-16T06:49:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001956667 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964344+0000", 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6174000000000004e-05 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964765+0000", 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042082000000000003 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965862+0000", 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0010968289999999999 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988875+0000", 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023013051 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.988969+0000", 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3350000000000006e-05 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:75 3.as0 3:5d679340:::smithi11731613-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.607251509000001, 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033077179999999998, 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 75 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962304+0000", 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962308+0000", 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0600000000000001e-06 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962314+0000", 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.088e-06 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962319+0000", 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1120000000000004e-06 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964978+0000", 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026582540000000001 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965065+0000", 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7108999999999996e-05 2024-09-16T06:49:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989270+0000", 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024205503 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989359+0000", 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9204999999999997e-05 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995382+0000", 2024-09-16T06:49:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060223200000000003 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:95 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.577366933, 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042547695000000003, 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 95 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992188+0000", 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992194+0000", 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8760000000000002e-06 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992204+0000", 2024-09-16T06:49:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.011e-05 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992212+0000", 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1890000000000001e-06 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993832+0000", 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001620565 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993886+0000", 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4021999999999999e-05 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994477+0000", 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059046499999999998 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995906+0000", 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014291919999999999 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034552+0000", 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038646106999999999 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034737+0000", 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018489900000000001 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:96 3.as0 3:5bc02ad9:::smithi11731613-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.57733795, 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.058410493000000001, 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 96 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992221+0000", 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2899999999999998e-06 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992229+0000", 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3080000000000008e-06 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992236+0000", 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.528e-06 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994995+0000", 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027588700000000001 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995064+0000", 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9758000000000005e-05 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035940+0000", 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040876066000000003 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036021+0000", 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0243999999999999e-05 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050629+0000", 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014608009999999999 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=18, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.574256202000001, 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040520444000000003, 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995299+0000", 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995304+0000", 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5680000000000001e-06 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034371+0000", 2024-09-16T06:49:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039067118999999997 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034380+0000", 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7949999999999996e-06 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034448+0000", 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8244000000000006e-05 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035820+0000", 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001372306 2024-09-16T06:49:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:144 3.as0 3:50fa89c4:::smithi11731613-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.505869732999997, 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38453364499999998, 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 144 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063691+0000", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8879999999999996e-06 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063699+0000", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5799999999999992e-06 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063705+0000", 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.246e-06 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065212+0000", 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015062820000000001 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065312+0000", 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010052600000000001 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065706+0000", 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039373300000000001 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070934+0000", 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052282070000000003 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448115+0000", 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37718074400000001 2024-09-16T06:49:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448220+0000", 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105104 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:145 3.as0 3:50fa89c4:::smithi11731613-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.505826763999998, 2024-09-16T06:49:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38866241499999998, 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 145 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063732+0000", 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1999999999999999e-06 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063741+0000", 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4700000000000008e-06 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063748+0000", 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4740000000000002e-06 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066083+0000", 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023351140000000001 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066155+0000", 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1650999999999996e-05 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448700+0000", 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38254526700000002 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448854+0000", 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154306 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.452392+0000", 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035374640000000001 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=23, last_complete=35'13, committed=1, applied=1))", 2024-09-16T06:49:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 62.498561553999998, 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37764362699999998, 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:25.607 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070994+0000", 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.071000+0000", 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8980000000000001e-06 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.608 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447985+0000", 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37698559799999998 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447996+0000", 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0672999999999999e-05 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448044+0000", 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8338999999999999e-05 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448638+0000", 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059397499999999995 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:302 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 2] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 55.434693406999997, 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21732169600000001, 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 302 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134859+0000", 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134878+0000", 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9264999999999999e-05 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134922+0000", 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3260000000000003e-05 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134944+0000", 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2603e-05 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135181+0000", 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023672400000000001 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135385+0000", 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020434300000000001 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.347993+0000", 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21260742799999999 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.351735+0000", 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037421830000000001 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352007+0000", 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027182200000000001 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352184+0000", 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017777400000000001 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:303 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 55.434587237000002, 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21552337099999999, 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 303 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134967+0000", 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134976+0000", 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3860000000000007e-06 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134996+0000", 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0234999999999999e-05 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135015+0000", 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8949000000000001e-05 2024-09-16T06:49:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135785+0000", 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000770388 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135925+0000", 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013959700000000001 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350066+0000", 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21414113400000001 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350271+0000", 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020513000000000001 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350492+0000", 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022106200000000001 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 61/32 ECSubWriteReply(tid=126, last_complete=61'99, committed=1, applied=1))", 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 36.693873599, 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63424799499999995, 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875681+0000", 2024-09-16T06:49:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875692+0000", 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1154e-05 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508662+0000", 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63297013099999999 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508675+0000", 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3217000000000001e-05 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508759+0000", 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3652999999999993e-05 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.509930+0000", 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001171486 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:542 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.741144905999999, 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031510007, 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 542 2024-09-16T06:49:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828399+0000", 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828425+0000", 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5700000000000001e-05 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828469+0000", 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3510999999999997e-05 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828493+0000", 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4769999999999998e-05 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828815+0000", 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032114900000000001 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828827+0000", 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2143999999999999e-05 2024-09-16T06:49:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835549+0000", 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.006721856 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835824+0000", 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027592399999999998 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.838011+0000", 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021865410000000002 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.849034+0000", 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011023141 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859755+0000", 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010721039999999999 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859921+0000", 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016604799999999999 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:543 3.6s0 3:64026cd9:::smithi11731613-35:head [rollback 24] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.74102667, 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034430898000000001, 2024-09-16T06:49:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 543 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828528+0000", 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828538+0000", 2024-09-16T06:49:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0457e-05 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828557+0000", 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8459000000000001e-05 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828571+0000", 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.447e-05 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828929+0000", 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035788200000000001 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828944+0000", 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4662e-05 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840136+0000", 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011191942 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840414+0000", 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027797100000000001 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.842038+0000", 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0016248160000000001 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855171+0000", 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013132853 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862552+0000", 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073803990000000002 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862960+0000", 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00040862400000000001 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:544 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.740972102000001, 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034849585000000002, 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 544 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828583+0000", 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828589+0000", 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5819999999999998e-06 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828607+0000", 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7260000000000001e-05 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828620+0000", 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3789e-05 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828991+0000", 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037102199999999999 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828999+0000", 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5000000000000002e-06 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843716+0000", 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0147174 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843970+0000", 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000253732 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845224+0000", 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001253524 2024-09-16T06:49:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855177+0000", 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0099530999999999994 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863271+0000", 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0080939529999999992 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863434+0000", 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016288899999999999 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:743 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 34] snapc 2e=[2e,2b,2a,29,28,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 14.724379374, 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028808912999999998, 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 743 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845175+0000", 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845184+0000", 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9590000000000001e-06 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845198+0000", 2024-09-16T06:49:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4282e-05 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845208+0000", 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0258000000000001e-05 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845266+0000", 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7373000000000001e-05 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845364+0000", 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8196000000000006e-05 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.869612+0000", 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024247713000000001 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.871840+0000", 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022288270000000001 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.873933+0000", 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020920740000000002 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.873986+0000", 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3121000000000002e-05 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/32 ECSubWrite(tid=148, reqid=client.4511.0:775, at_version=0'0, trim_to=0'0, roll_forward_to=86'141))", 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.614060932999999, 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.83265717400000006, 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955494+0000", 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955505+0000", 2024-09-16T06:49:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0801e-05 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786476+0000", 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.83097071499999997 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786491+0000", 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5109999999999999e-05 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786683+0000", 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000192165 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786704+0000", 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0873000000000002e-05 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.788074+0000", 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001370079 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.788152+0000", 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8248e-05 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:852 3.as0 3:57ffea71:::smithi11731613-30:head [append 0~0] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.6330565670000001, 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.029643135000000001, 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 852 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936496+0000", 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936509+0000", 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.205e-05 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936550+0000", 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1384000000000002e-05 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936564+0000", 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4053000000000001e-05 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936833+0000", 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026950699999999999 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936856+0000", 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2518999999999999e-05 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.941811+0000", 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00495547 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.941931+0000", 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011981699999999999 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943080+0000", 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001149129 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.965859+0000", 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022778732999999999 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966004+0000", 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014514199999999999 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966143+0000", 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000138391 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:853 3.as0 3:57ffea71:::smithi11731613-30:head [rollback 15] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.6329708410000001, 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.029774785000000002, 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 853 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936584+0000", 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936591+0000", 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.365e-06 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936603+0000", 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2191999999999999e-05 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936612+0000", 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0350000000000007e-06 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936994+0000", 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038190700000000001 2024-09-16T06:49:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937012+0000", 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8757e-05 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.947015+0000", 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010002341999999999 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.947373+0000", 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035835499999999998 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.950105+0000", 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027321899999999998 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966146+0000", 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016041161000000002 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966240+0000", 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3199e-05 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966360+0000", 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012037 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:854 3.as0 3:57ffea71:::smithi11731613-30:head [append 0~0] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.6329366140000001, 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.030019711000000001, 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 854 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936623+0000", 2024-09-16T06:49:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1659999999999996e-06 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936650+0000", 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7022e-05 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936658+0000", 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0809999999999994e-06 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937075+0000", 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000416628 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937091+0000", 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6586e-05 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.951259+0000", 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014167170999999999 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.951679+0000", 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042059200000000001 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.952960+0000", 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012813690000000001 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966424+0000", 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013463298 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966523+0000", 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9567000000000004e-05 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966639+0000", 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000115837 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s2 94/32 ECSubWrite(tid=166, reqid=osd.14.0:166, at_version=94'160, trim_to=0'0, roll_forward_to=92'156))", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.938487+0000", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.6310691369999999, 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027366186000000001, 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938487+0000", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938487+0000", 2024-09-16T06:49:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938485+0000", 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938529+0000", 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4076000000000003e-05 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938621+0000", 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1125999999999995e-05 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938651+0000", 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0193999999999999e-05 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938776+0000", 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000124721 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.938802+0000", 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6911999999999999e-05 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.965819+0000", 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027016476000000001 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.965853+0000", 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4257000000000001e-05 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:25.677 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:25.678 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:25.678 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:25.679 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:25.679 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:25.679 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:49:25.680 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:25.680 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:25.680 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:25.680+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:25.681 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:25.681 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:25.711 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:25.711 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:25.741 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:25.742 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:25.767 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:25.779 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:25.774+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:25.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:25.800+0000 7fd6627fc640 0 -- 172.21.15.100:0/657089801 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7fd66421c2f0 msgr2=0x7fd66421e6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:25.803 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:25.809 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:25.809 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:25.848 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:25.848 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:25.862 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:49:25.880 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:25.878+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:25.966 INFO:tasks.rados.rados.0.smithi117.stdout:341: done (4 left) 2024-09-16T06:49:25.966 INFO:tasks.rados.rados.0.smithi117.stdout:342: read oid 35 snap -1 2024-09-16T06:49:25.967 INFO:tasks.rados.rados.0.smithi117.stdout:342: expect (ObjNum 82 snap 11 seq_num 82) 2024-09-16T06:49:25.967 INFO:tasks.rados.rados.0.smithi117.stdout:343: snap_create 2024-09-16T06:49:25.967 INFO:tasks.rados.rados.0.smithi117.stdout:338: finishing copy_from racing read to smithi11731613-25 2024-09-16T06:49:25.967 INFO:tasks.rados.rados.0.smithi117.stdout:337: expect (ObjNum 121 snap 28 seq_num 121) 2024-09-16T06:49:25.981 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:25.981+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:25.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:49:25.997 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:25.998 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:26.013 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:49:26.040 INFO:tasks.rados.rados.0.smithi117.stdout:339: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T06:49:26.045 INFO:tasks.rados.rados.0.smithi117.stdout:338: finishing copy_from to smithi11731613-25 2024-09-16T06:49:26.045 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 25 v 200 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:26.045 INFO:tasks.rados.rados.0.smithi117.stdout:340: finishing copy_from racing read to smithi11731613-40 2024-09-16T06:49:26.046 INFO:tasks.rados.rados.0.smithi117.stdout:340: finishing copy_from to smithi11731613-40 2024-09-16T06:49:26.046 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 40 v 31 (ObjNum 82 snap 11 seq_num 82) dirty exists 2024-09-16T06:49:26.046 INFO:tasks.rados.rados.0.smithi117.stdout:342: expect (ObjNum 82 snap 11 seq_num 82) 2024-09-16T06:49:26.082 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:26.081+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:26.132 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.236717241999997, 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.236633445999999, 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:49:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:49:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.205228206999998, 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.205178439000001, 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:49:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:49:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.097440773999999, 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:49:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:49:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.096885315000002, 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:49:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.096828983999998, 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:49:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:219 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [copy-from ver 4] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.173609833999997, 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031057211000000001, 2024-09-16T06:49:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 219 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956214+0000", 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956234+0000", 2024-09-16T06:49:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0275000000000002e-05 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956284+0000", 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0219999999999997e-05 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956309+0000", 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4930999999999999e-05 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956442+0000", 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013270999999999999 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956459+0000", 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7317999999999999e-05 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960744+0000", 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0042850090000000002 2024-09-16T06:49:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961009+0000", 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026485300000000001 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.976584+0000", 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015575011999999999 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.981658+0000", 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050738470000000003 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987101+0000", 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054435580000000003 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987274+0000", 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017305700000000001 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:220 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.173487375999997, 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023931746, 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 220 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956338+0000", 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956351+0000", 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2849e-05 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956375+0000", 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4195000000000001e-05 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956391+0000", 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5352000000000001e-05 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956650+0000", 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025920099999999997 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956669+0000", 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9157999999999999e-05 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961818+0000", 2024-09-16T06:49:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051493709999999998 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961948+0000", 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012956 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979661+0000", 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017713421 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979887+0000", 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022550399999999999 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.980271+0000", 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038454100000000002 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:316 3.ds0 3:b2c197ab:::smithi11731613-17:head [copy-from ver 4] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 55.967811873000002, 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017020408000000001, 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 316 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162012+0000", 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162030+0000", 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8688000000000001e-05 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162067+0000", 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6526000000000002e-05 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162083+0000", 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6061000000000001e-05 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162190+0000", 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107351 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162511+0000", 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032124399999999999 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.174275+0000", 2024-09-16T06:49:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011763691999999999 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.176204+0000", 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019286329999999999 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.178962+0000", 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027583099999999999 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.179036+0000", 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3456999999999998e-05 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:317 3.ds0 3:b2c197ab:::smithi11731613-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 55.967726538000001, 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013682241, 2024-09-16T06:49:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 317 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162106+0000", 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4620000000000001e-06 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162124+0000", 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7706999999999998e-05 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162136+0000", 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2285e-05 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162962+0000", 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000826123 2024-09-16T06:49:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163038+0000", 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5371999999999996e-05 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175269+0000", 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01223169 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175524+0000", 2024-09-16T06:49:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025445300000000003 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175783+0000", 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025890899999999998 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:354 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 51.28728898, 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013614067000000001, 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 354 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842537+0000", 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842545+0000", 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2500000000000006e-06 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842562+0000", 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7512000000000001e-05 2024-09-16T06:49:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842573+0000", 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0976e-05 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844555+0000", 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001981919 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844621+0000", 2024-09-16T06:49:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5792999999999998e-05 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.856152+0000", 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011530883 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=96, reqid=osd.10.0:96, at_version=71'80, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.289092029999999, 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012334845000000001, 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840734+0000", 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840800+0000", 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5324e-05 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840986+0000", 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000186701 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841011+0000", 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4372e-05 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841156+0000", 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000145749 2024-09-16T06:49:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841215+0000", 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8845000000000003e-05 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853010+0000", 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011794392000000001 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853070+0000", 2024-09-16T06:49:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0158000000000002e-05 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=97, reqid=osd.10.0:97, at_version=71'82, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.289019519, 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013582399, 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840851+0000", 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4564999999999998e-05 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841065+0000", 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000213997 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841082+0000", 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6833e-05 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845856+0000", 2024-09-16T06:49:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0047738549999999996 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845879+0000", 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854322+0000", 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084429220000000003 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854390+0000", 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7514000000000004e-05 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 71/37 ECSubWrite(tid=20, reqid=osd.5.0:20, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.286664829999999, 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013186145, 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843211+0000", 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9116000000000002e-05 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843322+0000", 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000111142 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843344+0000", 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2350000000000001e-05 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848942+0000", 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005598235 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848970+0000", 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7875999999999999e-05 2024-09-16T06:49:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856277+0000", 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073063049999999999 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856348+0000", 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1741000000000003e-05 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds0 72/54 ECSubWrite(tid=156, reqid=osd.3.0:156, at_version=72'61, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.291935918, 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013858534, 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837890+0000", 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837941+0000", 2024-09-16T06:49:26.220 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0958999999999998e-05 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838125+0000", 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000184014 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838144+0000", 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8258999999999999e-05 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838277+0000", 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000133452 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838303+0000", 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6038999999999999e-05 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851693+0000", 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013389803 2024-09-16T06:49:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851750+0000", 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6641000000000002e-05 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 94/32 ECSubWrite(tid=260, reqid=client.4511.0:852, at_version=94'198, trim_to=0'0, roll_forward_to=92'196))", 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.1865428099999997, 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014393538000000001, 2024-09-16T06:49:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943283+0000", 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943342+0000", 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8780999999999998e-05 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943490+0000", 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014785699999999999 2024-09-16T06:49:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943516+0000", 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5681999999999999e-05 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944924+0000", 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00140785 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944950+0000", 2024-09-16T06:49:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6622e-05 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.957559+0000", 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012609120999999999 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.957678+0000", 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000118449 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 94/77 ECSubWrite(tid=160, reqid=osd.13.0:160, at_version=94'29, trim_to=0'0, roll_forward_to=92'27))", 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.1856457359999997, 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01233383, 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944271+0000", 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9946999999999998e-05 2024-09-16T06:49:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944400+0000", 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012965399999999999 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944414+0000", 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3454e-05 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944586+0000", 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000171893 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944603+0000", 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6923e-05 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956440+0000", 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011837640999999999 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956515+0000", 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4670000000000002e-05 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 94/32 ECSubWrite(tid=164, reqid=osd.10.0:164, at_version=94'136, trim_to=0'0, roll_forward_to=92'134))", 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.1854871239999998, 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012091431999999999, 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944367+0000", 2024-09-16T06:49:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7620999999999999e-05 2024-09-16T06:49:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944472+0000", 2024-09-16T06:49:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105237 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944482+0000", 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7540000000000001e-06 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944702+0000", 2024-09-16T06:49:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000220034 2024-09-16T06:49:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944716+0000", 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3465000000000001e-05 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956349+0000", 2024-09-16T06:49:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011633596 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956431+0000", 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2027000000000006e-05 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 94/32 ECSubWrite(tid=263, reqid=osd.3.0:263, at_version=94'202, trim_to=0'0, roll_forward_to=92'196))", 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.960919+0000", 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.1689074880000003, 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012030674999999999, 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.960919+0000", 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.960919+0000", 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.960917+0000", 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.960978+0000", 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.1008000000000003e-05 2024-09-16T06:49:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.961075+0000", 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6353000000000004e-05 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.961099+0000", 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4522e-05 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.961192+0000", 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3214999999999994e-05 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.961207+0000", 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4297999999999999e-05 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.972910+0000", 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011703237 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.972950+0000", 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0204999999999999e-05 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.244 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:26.246 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:49:26.246 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:26.246+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:26.347 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:26.352+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:26.415 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:26.416 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:26.429 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:49:26.474 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:26.474+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:26.548 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:26.546+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:26.551 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:26.552 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:26.565 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:49:26.649 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:26.649+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:26.732 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:26.732 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:26.732 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:26.732 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:26.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 35/32 ECSubWrite(tid=35, reqid=client.4511.0:152, at_version=35'10, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.653534299999997, 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.44133388099999998, 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077839+0000", 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077850+0000", 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0505e-05 2024-09-16T06:49:26.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515878+0000", 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.43802848300000002 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515883+0000", 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1780000000000002e-06 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.735 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515923+0000", 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9866000000000001e-05 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515930+0000", 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.224e-06 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519096+0000", 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031664449999999999 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519174+0000", 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7590999999999996e-05 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:163 3.cs0 3:31c9f1c0:::smithi11731613-46:head [copy-from ver 2 in=54b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.219292609999997, 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.80778171600000004, 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.737 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.738 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.738 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.738 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.738 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 163 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512085+0000", 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8870000000000002e-06 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512091+0000", 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3539999999999996e-06 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512096+0000", 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8029999999999996e-06 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513808+0000", 2024-09-16T06:49:26.741 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001712199 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513857+0000", 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8649000000000001e-05 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.524232+0000", 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010375422 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.527189+0000", 2024-09-16T06:49:26.742 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002956939 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319694+0000", 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79250512900000003 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319863+0000", 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168796 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.204665540999997, 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79400328899999995, 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.744 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:26.745 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526711+0000", 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.18e-06 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319361+0000", 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79264922400000004 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319384+0000", 2024-09-16T06:49:26.746 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3442e-05 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319508+0000", 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123683 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.320712+0000", 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012041 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.747 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.202690597, 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.394952786, 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:26.748 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:26.749 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528687+0000", 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0810000000000004e-06 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923559+0000", 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39487182199999998 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923572+0000", 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3406e-05 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923624+0000", 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1989e-05 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923636+0000", 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1990999999999999e-05 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=97, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 49.886574437, 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15568405900000001, 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:26.752 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844798+0000", 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.753 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844807+0000", 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8319999999999995e-06 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.998993+0000", 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15418552099999999 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999006+0000", 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3069e-05 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.754 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999149+0000", 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000142965 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000484+0000", 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013350300000000001 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 49/37 ECSubWriteReply(tid=98, last_complete=49'26, committed=1, applied=1))", 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 49.880863515000001, 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15060995699999999, 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:26.756 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850509+0000", 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.757 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850523+0000", 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4219000000000001e-05 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000937+0000", 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15041373399999999 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000960+0000", 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3768e-05 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.758 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001073+0000", 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112218 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001120+0000", 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7936999999999999e-05 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as6 49/32 ECSubWrite(tid=80, reqid=client.4511.0:373, at_version=49'61, trim_to=0'0, roll_forward_to=49'59))", 2024-09-16T06:49:26.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 49.799595216, 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.071706169, 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931779+0000", 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931778+0000", 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.931805+0000", 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7250000000000002e-05 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.761 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002139+0000", 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.070333802000000001 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002154+0000", 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5176e-05 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002259+0000", 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105357 2024-09-16T06:49:26.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002284+0000", 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4743e-05 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.003425+0000", 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001140605 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.003485+0000", 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0442999999999997e-05 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=99, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 49.729271498999999, 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123751468, 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.764 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002101+0000", 2024-09-16T06:49:26.765 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002109+0000", 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8399999999999995e-06 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125675+0000", 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123566417 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.766 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125711+0000", 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5215000000000001e-05 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125821+0000", 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110661 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125854+0000", 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2638000000000002e-05 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.767 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:428 3.cs0 3:31c9f1c0:::smithi11731613-46:head [delete] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 45.876472251000003, 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.62638514099999998, 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 428 2024-09-16T06:49:26.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854899+0000", 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854915+0000", 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5262999999999999e-05 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854948+0000", 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3005e-05 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.770 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854965+0000", 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7112000000000002e-05 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855070+0000", 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105198 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855081+0000", 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1308000000000001e-05 2024-09-16T06:49:26.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478008+0000", 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.622927122 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478196+0000", 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000187534 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.481287+0000", 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030910360000000001 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(osd.5.22:1 3.fs0 3:fa1c58b4:::smithi11731613-7:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 45.862694011000002, 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.61158470300000001, 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.773 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868678+0000", 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:26.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868690+0000", 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2336e-05 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868722+0000", 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2122000000000002e-05 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868744+0000", 2024-09-16T06:49:26.775 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1855e-05 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868876+0000", 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013210300000000001 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868888+0000", 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2053e-05 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.776 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478066+0000", 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.60917769200000005 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478207+0000", 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000140743 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.480265+0000", 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020579219999999998 2024-09-16T06:49:26.777 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:431 3.fs0 3:f3961c35:::smithi11731613-9:head [stat,delete] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 43.916393876000001, 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.106237357, 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.778 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 431 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814977+0000", 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814993+0000", 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6095000000000001e-05 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815025+0000", 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1822000000000001e-05 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815042+0000", 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6803000000000002e-05 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815181+0000", 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013884600000000001 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.781 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815482+0000", 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030067499999999998 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.817743+0000", 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022618769999999998 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.820149+0000", 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024051290000000002 2024-09-16T06:49:26.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921014+0000", 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.10086516700000001 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921218+0000", 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020381300000000001 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.783 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:583 3.fs0 3:fa1c58b4:::smithi11731613-7:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.672042549, 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41724843499999997, 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 583 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.784 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:26.785 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059336+0000", 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5140000000000001e-06 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059357+0000", 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0602e-05 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059374+0000", 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7640000000000001e-05 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062783+0000", 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003409081 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062871+0000", 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7188999999999995e-05 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.787 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.063266+0000", 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039578200000000001 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.064116+0000", 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00084932599999999997 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.788 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476470+0000", 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41235417099999999 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476580+0000", 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011006999999999999 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.789 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 72/37 ECSubWriteReply(tid=209, last_complete=72'94, committed=1, applied=1))", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.665814644000001, 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41187659199999999, 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:26.790 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065564+0000", 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2839999999999997e-06 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.791 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476221+0000", 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41065675400000001 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476236+0000", 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5509e-05 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.792 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476326+0000", 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9337000000000002e-05 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.477436+0000", 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011104210000000001 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.793 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.1s1 81/77 ECSubWrite(tid=85, reqid=client.4511.0:732, at_version=0'0, trim_to=0'0, roll_forward_to=81'47))", 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 17.424750573000001, 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38681583800000002, 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.794 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306621+0000", 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306649+0000", 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7733e-05 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.795 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692262+0000", 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.385613345 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692278+0000", 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5642e-05 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692421+0000", 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014278699999999999 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.796 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692441+0000", 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0089e-05 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693380+0000", 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00093900299999999999 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693439+0000", 2024-09-16T06:49:26.797 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9342000000000002e-05 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:796 3.cs0 3:3ac5e123:::smithi11731613-44:head [read 0~5816320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:15.891013+0000", 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.84036152, 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.050093192000000002, 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.798 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 796 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891013+0000", 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891013+0000", 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.799 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891012+0000", 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891022+0000", 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0776e-05 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891033+0000", 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0247e-05 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891049+0000", 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6152999999999999e-05 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891166+0000", 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000117142 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.801 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891177+0000", 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1197e-05 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.895807+0000", 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0046293760000000002 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.896111+0000", 2024-09-16T06:49:26.802 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030470999999999997 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.941106+0000", 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.044994460999999999 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:797 3.cs0 3:3ac5e123:::smithi11731613-44:head [read 0~5816320,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:15.891057+0000", 2024-09-16T06:49:26.803 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.840317024000001, 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.060193570000000002, 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 797 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891057+0000", 2024-09-16T06:49:26.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891057+0000", 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891056+0000", 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891061+0000", 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3830000000000002e-06 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891073+0000", 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2703e-05 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891082+0000", 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8599999999999999e-06 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891203+0000", 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012084800000000001 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.891209+0000", 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5539999999999997e-06 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.901320+0000", 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010111375000000001 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.901461+0000", 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000140476 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:15.951251+0000", 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.049790048000000003 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.808 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:813 3.cs0 3:3e2de01d:::smithi11731613-47:head [read 0~4947968,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:17.824484+0000", 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9068903119999998, 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.047242712999999999, 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.809 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 813 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824484+0000", 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824484+0000", 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824481+0000", 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824517+0000", 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6066999999999999e-05 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824552+0000", 2024-09-16T06:49:26.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5138999999999997e-05 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824570+0000", 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7776000000000001e-05 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824689+0000", 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000119564 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824705+0000", 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5849999999999999e-05 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.836682+0000", 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011976749 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.837142+0000", 2024-09-16T06:49:26.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00045970000000000001 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.871726+0000", 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034584885000000003 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:814 3.cs0 3:3e2de01d:::smithi11731613-47:head [read 0~4947968,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:17.824594+0000", 2024-09-16T06:49:26.814 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9067795870000008, 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.054826996000000003, 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 814 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824594+0000", 2024-09-16T06:49:26.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824594+0000", 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824593+0000", 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.816 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824600+0000", 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3459999999999999e-06 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824615+0000", 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5119999999999999e-05 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824627+0000", 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2546e-05 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824857+0000", 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022972199999999999 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824886+0000", 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9200000000000002e-05 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.843393+0000", 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018506246000000001 2024-09-16T06:49:26.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.843650+0000", 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025721099999999997 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.879421+0000", 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.035771575 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:815 3.cs0 3:3e2de01d:::smithi11731613-47:head [read 0~4947968,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:17.824677+0000", 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9066972530000008, 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.060889908999999999, 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 815 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824677+0000", 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824677+0000", 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824676+0000", 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824684+0000", 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6999999999999997e-06 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824704+0000", 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9453000000000002e-05 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824718+0000", 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3949e-05 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824965+0000", 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00024782800000000001 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.824983+0000", 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8006999999999999e-05 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.844336+0000", 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.019352461000000001 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.844914+0000", 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00057832900000000004 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:17.885567+0000", 2024-09-16T06:49:26.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040652418000000003 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:857 3.cs0 3:3e2de01d:::smithi11731613-47:head [read 0~4947968,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.937328+0000", 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 4.7940457509999996, 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.052477638, 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:26.825 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 857 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937328+0000", 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937328+0000", 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:26.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937327+0000", 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937354+0000", 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6789000000000001e-05 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:26.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937410+0000", 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6141999999999999e-05 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937426+0000", 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6622000000000001e-05 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937568+0000", 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014180700000000001 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937591+0000", 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2365999999999999e-05 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.942837+0000", 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052463589999999999 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.942983+0000", 2024-09-16T06:49:26.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014641900000000001 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.989806+0000", 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.046822613999999999 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:26.830 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:26.832 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:49:26.832 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:26.836+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:26.932 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:26.936+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:26.954 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:26.954 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:26.954 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:26.954 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:26.965 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:49:26.968 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:337: done (5 left) 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:338: done (4 left) 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:339: done (3 left) 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:340: done (2 left) 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:342: done (1 left) 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:343: done (0 left) 2024-09-16T06:49:26.969 INFO:tasks.rados.rados.0.smithi117.stdout:344: snap_create 2024-09-16T06:49:27.033 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:27.036+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:27.089 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:27.090 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:27.104 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:49:27.153 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:27.154+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:266 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:27.232 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.275881189000003, 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014774862999999999, 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 266 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:27.233 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953825+0000", 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.234 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953846+0000", 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0171999999999999e-05 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953889+0000", 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3284000000000001e-05 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953913+0000", 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4071e-05 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954055+0000", 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014240699999999999 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954076+0000", 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0678000000000001e-05 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961148+0000", 2024-09-16T06:49:27.236 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070716959999999997 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961402+0000", 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025470100000000003 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.968604+0000", 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072013340000000002 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.237 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:267 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.275764977000001, 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015485804000000001, 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 267 2024-09-16T06:49:27.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953944+0000", 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953973+0000", 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9581e-05 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953995+0000", 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1685e-05 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954011+0000", 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5862000000000002e-05 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954223+0000", 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021187199999999999 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954233+0000", 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0789e-05 2024-09-16T06:49:27.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962540+0000", 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0083068819999999998 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962710+0000", 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017007799999999999 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.969431+0000", 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0067204250000000004 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:268 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.275684730000002, 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.30266284300000001, 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:27.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 268 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:27.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954024+0000", 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954032+0000", 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4290000000000004e-06 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954050+0000", 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8332e-05 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954063+0000", 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2955999999999999e-05 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954379+0000", 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031616000000000001 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954389+0000", 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3959999999999995e-06 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963428+0000", 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0090396959999999998 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963723+0000", 2024-09-16T06:49:27.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00029431800000000001 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:29.256688+0000", 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.29296564200000003 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 50.394422540999997, 2024-09-16T06:49:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014247134, 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835286+0000", 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835349+0000", 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3258999999999999e-05 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835459+0000", 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110097 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835480+0000", 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1424999999999999e-05 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835600+0000", 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011922 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835623+0000", 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3144999999999999e-05 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849454+0000", 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013830779 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849535+0000", 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1021999999999998e-05 2024-09-16T06:49:27.252 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 61/32 ECSubWrite(tid=83, reqid=osd.14.0:83, at_version=61'72, trim_to=0'0, roll_forward_to=59'68))", 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 38.396137363999998, 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012777472999999999, 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.253 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833571+0000", 2024-09-16T06:49:27.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833631+0000", 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9917999999999998e-05 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833761+0000", 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013055799999999999 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833784+0000", 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2714999999999999e-05 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835926+0000", 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021422720000000002 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835943+0000", 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6889e-05 2024-09-16T06:49:27.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846281+0000", 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010337641999999999 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846350+0000", 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9315999999999998e-05 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 61/32 ECSubWrite(tid=85, reqid=osd.10.0:85, at_version=61'70, trim_to=0'0, roll_forward_to=59'66))", 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 38.391255631, 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013138069, 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838452+0000", 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838506+0000", 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3594999999999998e-05 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839270+0000", 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00076359400000000003 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839293+0000", 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3006e-05 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839420+0000", 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127355 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839446+0000", 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851525+0000", 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012079196 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851592+0000", 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7156999999999994e-05 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=101, reqid=osd.10.0:101, at_version=72'86, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.392242437, 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017459348, 2024-09-16T06:49:27.262 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.263 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837467+0000", 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837538+0000", 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0541000000000001e-05 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837664+0000", 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012650899999999999 2024-09-16T06:49:27.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837687+0000", 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2759999999999999e-05 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839677+0000", 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019901300000000001 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839707+0000", 2024-09-16T06:49:27.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0049e-05 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854862+0000", 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015155366 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854927+0000", 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4629999999999996e-05 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=157, reqid=osd.3.0:157, at_version=72'123, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.391929483999998, 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027234286, 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837780+0000", 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837839+0000", 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8801000000000002e-05 2024-09-16T06:49:27.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837959+0000", 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000120436 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837982+0000", 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3156e-05 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838118+0000", 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000135673 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838143+0000", 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4899e-05 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.864948+0000", 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026805539999999999 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.270 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865015+0000", 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6501000000000006e-05 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=102, reqid=osd.10.0:102, at_version=72'88, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.389118499999999, 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018351104, 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.271 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.272 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840590+0000", 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840640+0000", 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0631999999999999e-05 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840728+0000", 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7933e-05 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840750+0000", 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2022999999999999e-05 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840864+0000", 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113423 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840889+0000", 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5092999999999998e-05 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858883+0000", 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017994231999999999 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858943+0000", 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9587999999999999e-05 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.275 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=158, reqid=osd.3.0:158, at_version=72'125, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.387477362999999, 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022836532999999999, 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.276 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842231+0000", 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842295+0000", 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4238000000000004e-05 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.277 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842418+0000", 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012244299999999999 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842441+0000", 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3606000000000001e-05 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850726+0000", 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082850219999999995 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850750+0000", 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3822e-05 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865034+0000", 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014283563000000001 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865069+0000", 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5712e-05 2024-09-16T06:49:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=65, reqid=osd.12.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.384154337999998, 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017333787, 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.280 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845555+0000", 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.281 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845609+0000", 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4284999999999999e-05 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845766+0000", 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015658100000000001 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845789+0000", 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2866000000000001e-05 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845942+0000", 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015326200000000001 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845969+0000", 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7296000000000001e-05 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862827+0000", 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016857351999999999 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862890+0000", 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2997999999999996e-05 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=66, reqid=osd.12.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.382462852, 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017965090999999999, 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.284 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847246+0000", 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847298+0000", 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2677999999999999e-05 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847412+0000", 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113403 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.286 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847437+0000", 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4973e-05 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857022+0000", 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0095850889999999998 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857045+0000", 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3796000000000001e-05 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865168+0000", 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0081221260000000003 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865212+0000", 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4691999999999998e-05 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=108, reqid=osd.14.0:108, at_version=72'100, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.37936384, 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015047932999999999, 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850339+0000", 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999928 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850413+0000", 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3508000000000002e-05 2024-09-16T06:49:27.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850503+0000", 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0303999999999995e-05 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850526+0000", 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2595999999999998e-05 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850694+0000", 2024-09-16T06:49:27.291 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168367 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850718+0000", 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.44e-05 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865344+0000", 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014625658999999999 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.292 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865394+0000", 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0074999999999997e-05 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=109, reqid=osd.14.0:109, at_version=72'102, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.375716271999998, 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013551083, 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.293 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853992+0000", 2024-09-16T06:49:27.294 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854050+0000", 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8495000000000002e-05 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854252+0000", 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020159999999999999 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.295 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854296+0000", 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4438999999999999e-05 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862592+0000", 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082959690000000003 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862662+0000", 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9210999999999999e-05 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.296 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867494+0000", 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048324029999999999 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867545+0000", 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0838999999999999e-05 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.297 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:559 3.5s0 3:af79444c:::smithi11731613-29:1d [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 27.274730114, 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014695526, 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 559 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.298 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954979+0000", 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.299 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954984+0000", 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5310000000000003e-06 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954994+0000", 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7249999999999992e-06 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.955000+0000", 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7569999999999998e-06 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.300 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967068+0000", 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012068096 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967239+0000", 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017012400000000001 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.969675+0000", 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024368660000000002 2024-09-16T06:49:27.301 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=192, reqid=osd.6.0:192, at_version=73'33, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.380924028999999, 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017679872999999999, 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:27.302 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.303 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.303 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.303 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:27.303 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848785+0000", 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848820+0000", 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4931999999999997e-05 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.304 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849087+0000", 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026683499999999999 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849103+0000", 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6813e-05 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855706+0000", 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066022570000000003 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855730+0000", 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4661999999999999e-05 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866426+0000", 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010695893999999999 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866466+0000", 2024-09-16T06:49:27.306 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9632999999999998e-05 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.378798127, 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015506908999999999, 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.307 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850911+0000", 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850952+0000", 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0722000000000001e-05 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851117+0000", 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000164599 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.309 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851132+0000", 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5255e-05 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854555+0000", 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034234320000000001 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854579+0000", 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866327+0000", 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011748364000000001 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866419+0000", 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1624000000000002e-05 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.311 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 73/32 ECSubWrite(tid=107, reqid=osd.10.0:107, at_version=73'90, trim_to=0'0, roll_forward_to=72'88))", 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.375545656, 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012199229000000001, 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.312 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854217+0000", 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3168000000000003e-05 2024-09-16T06:49:27.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854327+0000", 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109833 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854349+0000", 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2296000000000001e-05 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855253+0000", 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090369500000000004 2024-09-16T06:49:27.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855283+0000", 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9884999999999999e-05 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866242+0000", 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01095932 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866364+0000", 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000121542 2024-09-16T06:49:27.315 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:705 3.5s0 3:af79444c:::smithi11731613-29:head [append 0~524288] snapc 2b=[2b,2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 21.195974081999999, 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012998625, 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:27.316 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 705 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.317 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033733+0000", 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034535+0000", 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00080219500000000001 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034573+0000", 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8473000000000002e-05 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.318 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034605+0000", 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1776999999999997e-05 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034755+0000", 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000149641 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.035008+0000", 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025373600000000001 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.039658+0000", 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0046500159999999999 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046264+0000", 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066053680000000004 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046596+0000", 2024-09-16T06:49:27.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000331983 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046735+0000", 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013881599999999999 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 94/32 ECSubWrite(tid=165, reqid=osd.14.0:165, at_version=94'158, trim_to=0'0, roll_forward_to=92'156))", 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 5.2940202960000002, 2024-09-16T06:49:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012800693, 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935689+0000", 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935746+0000", 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7460999999999998e-05 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935852+0000", 2024-09-16T06:49:27.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105815 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935875+0000", 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2886000000000001e-05 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936046+0000", 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000170929 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936070+0000", 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4746e-05 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.948119+0000", 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012048432 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.948490+0000", 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037153400000000002 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.325 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.326 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:27.326 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:27.326 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:27.327 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:27.326+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:27.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:49:27.427 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:27.426+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:27.459 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:27.459 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:27.459 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:27.459 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:27.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:49:27.529 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:27.532+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:27.596 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 30, 2024-09-16T06:49:27.596 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-16T06:49:27.596 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:27.610 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:49:27.630 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:27.630+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:27.731 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:27.736+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:135 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'93, mlcod=25'89)", 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.300932868000004, 2024-09-16T06:49:27.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073860260000000004, 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476338+0000", 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476337+0000", 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476374+0000", 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7509000000000003e-05 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476453+0000", 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8273000000000001e-05 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476468+0000", 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.499e-05 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476574+0000", 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000106879 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.476624+0000", 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9910000000000002e-05 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483640+0000", 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00701565 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.483724+0000", 2024-09-16T06:49:27.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4535000000000003e-05 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:141 1.0 e25/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'99, mlcod=25'91)", 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.291429061000002, 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.28011762800000001, 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485842+0000", 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485841+0000", 2024-09-16T06:49:27.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.485852+0000", 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1313e-05 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764267+0000", 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.278414628 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764288+0000", 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.105e-05 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764448+0000", 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016007599999999999 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.764487+0000", 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9125999999999998e-05 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765872+0000", 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001384763 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.765960+0000", 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7979000000000006e-05 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:161 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'115, mlcod=26'107)", 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.847651288999998, 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062540549999999997, 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929620+0000", 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929715+0000", 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5147000000000002e-05 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929991+0000", 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027581399999999999 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929997+0000", 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1399999999999997e-06 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932071+0000", 2024-09-16T06:49:27.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020736499999999998 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.932100+0000", 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9090000000000001e-05 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935772+0000", 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0036722019999999998 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935874+0000", 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102296 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:163 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'117, mlcod=26'107)", 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.847474196999997, 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062254820000000001, 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.788 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929797+0000", 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929801+0000", 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2030000000000002e-06 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930091+0000", 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028969100000000002 2024-09-16T06:49:27.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930097+0000", 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1870000000000002e-06 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933566+0000", 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003469663 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:27.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.933600+0000", 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3293000000000001e-05 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935942+0000", 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023424190000000001 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936023+0000", 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0438999999999993e-05 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:164 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'118, mlcod=26'107)", 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.847467753999993, 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0062774559999999998, 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:49:27.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929804+0000", 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929803+0000", 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929846+0000", 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3038999999999999e-05 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930109+0000", 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00026273799999999999 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930115+0000", 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0900000000000001e-06 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934327+0000", 2024-09-16T06:49:27.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004212248 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.934363+0000", 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5576e-05 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936028+0000", 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016651770000000001 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936081+0000", 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2874000000000002e-05 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4185.0:165 1.0 e26/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'119, mlcod=26'107)", 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.847422011000006, 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0069224810000000003, 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929849+0000", 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.929854+0000", 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7600000000000002e-06 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930127+0000", 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000273374 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.930135+0000", 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.157e-06 2024-09-16T06:49:27.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935309+0000", 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051733370000000001 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.935340+0000", 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1423000000000001e-05 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936696+0000", 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013555730000000001 2024-09-16T06:49:27.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:09.936772+0000", 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6260000000000005e-05 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=32, reqid=client.4511.0:237, at_version=40'12, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.815146026000001, 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.008178467, 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.802 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962125+0000", 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962124+0000", 2024-09-16T06:49:27.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962148+0000", 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3572999999999999e-05 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962287+0000", 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013903200000000001 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962295+0000", 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1999999999999994e-06 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962430+0000", 2024-09-16T06:49:27.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000135129 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962474+0000", 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3503e-05 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970216+0000", 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077425230000000003 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7330000000000006e-05 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=34, reqid=client.4511.0:228, at_version=40'13, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.814397296000003, 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075127500000000003, 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962874+0000", 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962872+0000", 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.962920+0000", 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7939999999999998e-05 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963011+0000", 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0655999999999994e-05 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963034+0000", 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3782999999999999e-05 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964104+0000", 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010698509999999999 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964127+0000", 2024-09-16T06:49:27.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2563000000000001e-05 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970304+0000", 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061773169999999999 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970387+0000", 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2680000000000001e-05 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.811 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=35, reqid=client.4511.0:238, at_version=40'13, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.813965623000001, 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00704997, 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963306+0000", 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963304+0000", 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963342+0000", 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7753000000000003e-05 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963421+0000", 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9549999999999996e-05 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963436+0000", 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4885999999999999e-05 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964389+0000", 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095243900000000002 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964403+0000", 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.381e-05 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970320+0000", 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059176259999999996 2024-09-16T06:49:27.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970356+0000", 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5525999999999998e-05 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 40/37 ECSubWrite(tid=36, reqid=client.4511.0:229, at_version=40'14, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.813457333999999, 2024-09-16T06:49:27.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006602887, 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963814+0000", 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963812+0000", 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963860+0000", 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7182999999999999e-05 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963954+0000", 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3912999999999993e-05 2024-09-16T06:49:27.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.963975+0000", 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1506e-05 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964846+0000", 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087055200000000002 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964868+0000", 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.243e-05 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970396+0000", 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005528397 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970417+0000", 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0418999999999999e-05 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.820 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/37 ECSubWrite(tid=37, reqid=client.4511.0:239, at_version=40'14, trim_to=0'0, roll_forward_to=38'10))", 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.813263382000002, 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070022000000000001, 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964008+0000", 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964007+0000", 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964083+0000", 2024-09-16T06:49:27.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6262999999999998e-05 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964162+0000", 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8568999999999999e-05 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.964181+0000", 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9283000000000002e-05 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965031+0000", 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00084985599999999996 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.965056+0000", 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4868999999999999e-05 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.970969+0000", 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059132270000000001 2024-09-16T06:49:27.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:27.971010+0000", 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1285999999999997e-05 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=80, reqid=osd.1.0:78, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 47.897140407000002, 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064748710000000001, 2024-09-16T06:49:27.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880131+0000", 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880130+0000", 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880152+0000", 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2322000000000001e-05 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880228+0000", 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6367999999999998e-05 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880245+0000", 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6186999999999999e-05 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880330+0000", 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.5568000000000005e-05 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.880347+0000", 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6528000000000002e-05 2024-09-16T06:49:27.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886553+0000", 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006206423 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.886606+0000", 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2704999999999998e-05 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/37 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'37, trim_to=0'0, roll_forward_to=50'35))", 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 47.895593966, 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0081001449999999996, 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881677+0000", 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881676+0000", 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.999999 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881715+0000", 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8512e-05 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881760+0000", 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4669999999999998e-05 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881776+0000", 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6371999999999998e-05 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881968+0000", 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00019165999999999999 2024-09-16T06:49:27.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.881998+0000", 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0332999999999999e-05 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889735+0000", 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077367310000000002 2024-09-16T06:49:27.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.889778+0000", 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2766999999999999e-05 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 52/37 ECSubWrite(tid=81, reqid=osd.1.0:81, at_version=52'42, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 47.894612582000001, 2024-09-16T06:49:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076324610000000001, 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882659+0000", 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882657+0000", 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882697+0000", 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9900000000000001e-05 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882763+0000", 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5949000000000002e-05 2024-09-16T06:49:27.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.882779+0000", 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6118000000000001e-05 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883074+0000", 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000294393 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.883098+0000", 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4049999999999998e-05 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890245+0000", 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071470479999999996 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:39.890291+0000", 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6343e-05 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/32 ECSubWrite(tid=177, reqid=osd.6.0:177, at_version=61'18, trim_to=0'0, roll_forward_to=57'16))", 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.819837+0000", 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.95743444, 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078136230000000004, 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819837+0000", 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819837+0000", 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819836+0000", 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819882+0000", 2024-09-16T06:49:27.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6022000000000001e-05 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.819984+0000", 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010180599999999999 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820009+0000", 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4746e-05 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820373+0000", 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036459699999999998 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.820399+0000", 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6060000000000001e-05 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.827615+0000", 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072153490000000002 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.827651+0000", 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5709e-05 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.4s6 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:48:48.823369+0000", 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.953902075000002, 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0086286320000000007, 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.843 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823369+0000", 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823369+0000", 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823368+0000", 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823392+0000", 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4474999999999999e-05 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823437+0000", 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4465999999999997e-05 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823447+0000", 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0073e-05 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823566+0000", 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118724 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.823595+0000", 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9471999999999998e-05 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831950+0000", 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0083552839999999993 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:48:48.831998+0000", 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7636000000000003e-05 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.847 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 94/77 ECSubWrite(tid=161, reqid=osd.13.0:161, at_version=94'65, trim_to=0'0, roll_forward_to=92'63))", 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:21.946819+0000", 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.8304520049999997, 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0067346929999999999, 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.946819+0000", 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.946819+0000", 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.946819+0000", 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.946879+0000", 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0482999999999997e-05 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.948886+0000", 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020064029999999999 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.948928+0000", 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2055999999999998e-05 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.950541+0000", 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016131660000000001 2024-09-16T06:49:27.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.950580+0000", 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9660000000000003e-05 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.953502+0000", 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029216020000000001 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.953554+0000", 2024-09-16T06:49:27.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1946e-05 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 94/77 ECSubWrite(tid=163, reqid=client.4511.0:849, at_version=94'69, trim_to=0'0, roll_forward_to=92'63))", 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:21.950135+0000", 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.8271362360000003, 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064673380000000004, 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.950135+0000", 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.950135+0000", 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.950133+0000", 2024-09-16T06:49:27.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.950187+0000", 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3646000000000001e-05 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.953770+0000", 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035831600000000002 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.953792+0000", 2024-09-16T06:49:27.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2393000000000001e-05 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.953912+0000", 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119546 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.953935+0000", 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.956495+0000", 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025599780000000001 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.956602+0000", 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010707199999999999 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 94/77 ECSubWrite(tid=165, reqid=client.4511.0:851, at_version=94'71, trim_to=0'0, roll_forward_to=92'63))", 2024-09-16T06:49:27.856 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:21.954811+0000", 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.8224600429999995, 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.014737449, 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.954811+0000", 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.954811+0000", 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.954810+0000", 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.954851+0000", 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.083e-05 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.968085+0000", 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013234727999999999 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.968107+0000", 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2086000000000002e-05 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.968265+0000", 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000157799 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.968295+0000", 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9635999999999999e-05 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.969488+0000", 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011932399999999999 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.969549+0000", 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0673e-05 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.860 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "MOSDECSubOpWrite(3.1s5 94/77 ECSubWrite(tid=170, reqid=client.4511.0:851, at_version=0'0, trim_to=0'0, roll_forward_to=94'71))", 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-16T06:49:21.971977+0000", 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 5.8052941410000001, 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.013487413, 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.971977+0000", 2024-09-16T06:49:27.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.971977+0000", 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.971975+0000", 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-16T06:49:27.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.972001+0000", 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5919999999999999e-05 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.983418+0000", 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.01141638 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.983442+0000", 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3977000000000001e-05 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.983575+0000", 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000133232 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_started", 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.983604+0000", 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9476e-05 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_committed", 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.985416+0000", 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018114680000000001 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-16T06:49:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-16T06:49:21.985465+0000", 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8816999999999997e-05 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-16T06:49:27.865 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:27.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:27.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:49:27.868 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:27.872+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:27.968 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:27.966+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:344: done (0 left) 2024-09-16T06:49:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:345: rollback oid 27 current snap is 38 2024-09-16T06:49:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 27 to 24 2024-09-16T06:49:27.978 INFO:tasks.rados.rados.0.smithi117.stdout:346: delete oid 39 current snap is 38 2024-09-16T06:49:27.989 INFO:tasks.rados.rados.0.smithi117.stdout:345: finishing rollback tid 0 to smithi11731613-27 2024-09-16T06:49:27.990 INFO:tasks.rados.rados.0.smithi117.stdout:345: finishing rollback tid 1 to smithi11731613-27 2024-09-16T06:49:27.990 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 27 v 181 (ObjNum 1681 snap 0 seq_num 4097581424) dirty dne 2024-09-16T06:49:27.991 INFO:tasks.rados.rados.0.smithi117.stdout:345: done (1 left) 2024-09-16T06:49:27.991 INFO:tasks.rados.rados.0.smithi117.stdout:346: done (0 left) 2024-09-16T06:49:27.991 INFO:tasks.rados.rados.0.smithi117.stdout:347: snap_remove snap 18 2024-09-16T06:49:28.035 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:49:28.036 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-16T06:49:28.036 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-16T06:49:28.036 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:49:28.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:49:28.068 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:28.068 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:28.069 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:28.066+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:28.093 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:28.093 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:28.118 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:28.118 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:28.143 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:28.163 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:28.163 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:28.166 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:28.166 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:28.169 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:28.172+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:28.181 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:49:28.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:28.188 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:28.213 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:28.234 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:28.234 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:28.259 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:28.259 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:28.270 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:28.266+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:28.286 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:28.286 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:28.309 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:28.309 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:28.314 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:28.314 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:28.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:49:28.338 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:28.338 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:28.366 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:28.366 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:28.371 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:28.370+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:28.394 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:28.394 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:28.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:28.404+0000 7f2a32ffd640 0 -- 172.21.15.100:0/905689295 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f2a3421f580 msgr2=0x7f2a34221970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:28.428 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:28.428 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:28.432 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:28.432 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:28.454 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:28.454 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:28.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:49:28.472 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:28.470+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:28.572 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:28.570+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:28.634 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:28.634 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:28.648 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:49:28.673 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:28.670+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:28.770 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:28.771 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:28.774 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:28.773+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:28.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:49:28.875 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:28.873+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:28.901 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:28.901 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:28.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:49:28.976 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:28.974+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:28.986 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 13] out_osds: [] dead_osds: [5] live_osds: [0, 4, 8, 12, 1, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:49:28.986 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:49:28.986 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T06:49:28.986 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-16T06:49:28.987 INFO:tasks.rados.rados.0.smithi117.stdout:347: done (0 left) 2024-09-16T06:49:28.987 INFO:tasks.rados.rados.0.smithi117.stdout:348: setattr oid 12 current snap is 38 2024-09-16T06:49:28.989 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T06:49:28.990 INFO:tasks.rados.rados.0.smithi117.stdout:349: delete oid 33 current snap is 38 2024-09-16T06:49:29.013 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 12 v 171 (ObjNum 289 snap 0 seq_num 957524701) dirty exists 2024-09-16T06:49:29.043 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.043 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:49:29.077 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:29.078+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:29.178 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:29.174+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:29.222 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.222 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.236 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:49:29.279 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:29.278+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:29.370 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.370 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.379 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:29.378+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:29.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:49:29.481 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:29.478+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:29.506 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.506 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:348: done (1 left) 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:349: done (0 left) 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:350: copy_from oid 7 from oid 25 current snap is 38 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:351: read oid 32 snap 14 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:351: expect (ObjNum 77 snap 11 seq_num 77) 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 10 current snap is 38 2024-09-16T06:49:29.526 INFO:tasks.rados.rados.0.smithi117.stdout:352: seq_num 141 ranges {2408448=425984,2834432=573440,3407872=425984,3833856=524288,4358144=425984} 2024-09-16T06:49:29.573 INFO:tasks.rados.rados.0.smithi117.stdout:352: writing smithi11731613-10 from 2408448 to 2834432 tid 1 2024-09-16T06:49:29.582 INFO:tasks.rados.rados.0.smithi117.stdout:352: writing smithi11731613-10 from 2834432 to 3407872 tid 2 2024-09-16T06:49:29.582 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:29.584+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:29.588 INFO:tasks.rados.rados.0.smithi117.stdout:352: writing smithi11731613-10 from 3407872 to 3833856 tid 3 2024-09-16T06:49:29.595 INFO:tasks.rados.rados.0.smithi117.stdout:352: writing smithi11731613-10 from 3833856 to 4358144 tid 4 2024-09-16T06:49:29.600 INFO:tasks.rados.rados.0.smithi117.stdout:352: writing smithi11731613-10 from 4358144 to 4784128 tid 5 2024-09-16T06:49:29.600 INFO:tasks.rados.rados.0.smithi117.stdout:353: read oid 26 snap -1 2024-09-16T06:49:29.601 INFO:tasks.rados.rados.0.smithi117.stdout:353: expect deleted 2024-09-16T06:49:29.601 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 2 current snap is 38 2024-09-16T06:49:29.601 INFO:tasks.rados.rados.0.smithi117.stdout:354: seq_num 142 ranges {32768=442368,475136=720896,1196032=540672,1736704=688128} 2024-09-16T06:49:29.605 INFO:tasks.rados.rados.0.smithi117.stdout:354: writing smithi11731613-2 from 32768 to 475136 tid 1 2024-09-16T06:49:29.613 INFO:tasks.rados.rados.0.smithi117.stdout:354: writing smithi11731613-2 from 475136 to 1196032 tid 2 2024-09-16T06:49:29.618 INFO:tasks.rados.rados.0.smithi117.stdout:354: writing smithi11731613-2 from 1196032 to 1736704 tid 3 2024-09-16T06:49:29.624 INFO:tasks.rados.rados.0.smithi117.stdout:354: writing smithi11731613-2 from 1736704 to 2424832 tid 4 2024-09-16T06:49:29.624 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 42 current snap is 38 2024-09-16T06:49:29.624 INFO:tasks.rados.rados.0.smithi117.stdout:351: expect (ObjNum 77 snap 11 seq_num 77) 2024-09-16T06:49:29.640 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.640 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:49:29.682 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:29.684+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:29.721 INFO:tasks.rados.rados.0.smithi117.stdout:350: finishing copy_from racing read to smithi11731613-7 2024-09-16T06:49:29.721 INFO:tasks.rados.rados.0.smithi117.stdout:350: finishing copy_from to smithi11731613-7 2024-09-16T06:49:29.721 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 7 v 200 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 1 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 2 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 3 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 4 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:354: finishing write tid 1 to smithi11731613-2 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 5 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 6 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: finishing write tid 7 to smithi11731613-10 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: oid 10 updating version 0 to 241 2024-09-16T06:49:29.722 INFO:tasks.rados.rados.0.smithi117.stdout:352: oid 10 version 241 is already newer than 240 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:352: oid 10 version 241 is already newer than 237 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:352: oid 10 version 241 is already newer than 236 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:352: oid 10 version 241 is already newer than 238 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:352: oid 10 version 241 is already newer than 239 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 10 v 241 (ObjNum 141 snap 38 seq_num 141) dirty exists 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:352: left oid 10 (ObjNum 141 snap 38 seq_num 141) 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: finishing write tid 2 to smithi11731613-2 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: finishing write tid 3 to smithi11731613-2 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: finishing write tid 4 to smithi11731613-2 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: finishing write tid 5 to smithi11731613-2 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: finishing write tid 6 to smithi11731613-2 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: oid 2 updating version 0 to 191 2024-09-16T06:49:29.723 INFO:tasks.rados.rados.0.smithi117.stdout:354: oid 2 updating version 191 to 195 2024-09-16T06:49:29.724 INFO:tasks.rados.rados.0.smithi117.stdout:354: oid 2 version 195 is already newer than 192 2024-09-16T06:49:29.724 INFO:tasks.rados.rados.0.smithi117.stdout:354: oid 2 version 195 is already newer than 193 2024-09-16T06:49:29.724 INFO:tasks.rados.rados.0.smithi117.stdout:354: oid 2 version 195 is already newer than 194 2024-09-16T06:49:29.724 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 2 v 195 (ObjNum 142 snap 38 seq_num 142) dirty exists 2024-09-16T06:49:29.724 INFO:tasks.rados.rados.0.smithi117.stdout:354: left oid 2 (ObjNum 142 snap 38 seq_num 142) 2024-09-16T06:49:29.724 INFO:tasks.rados.rados.0.smithi117.stdout:355: seq_num 143 ranges {1671168=458752,2129920=475136,2605056=655360,3260416=671744,3932160=147456} 2024-09-16T06:49:29.743 INFO:tasks.rados.rados.0.smithi117.stdout:355: writing smithi11731613-42 from 1671168 to 2129920 tid 1 2024-09-16T06:49:29.748 INFO:tasks.rados.rados.0.smithi117.stdout:355: writing smithi11731613-42 from 2129920 to 2605056 tid 2 2024-09-16T06:49:29.756 INFO:tasks.rados.rados.0.smithi117.stdout:355: writing smithi11731613-42 from 2605056 to 3260416 tid 3 2024-09-16T06:49:29.765 INFO:tasks.rados.rados.0.smithi117.stdout:355: writing smithi11731613-42 from 3260416 to 3932160 tid 4 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:355: writing smithi11731613-42 from 3932160 to 4079616 tid 5 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:350: done (5 left) 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:351: done (4 left) 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:352: done (3 left) 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:353: done (2 left) 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:354: done (1 left) 2024-09-16T06:49:29.767 INFO:tasks.rados.rados.0.smithi117.stdout:356: read oid 38 snap 33 2024-09-16T06:49:29.768 INFO:tasks.rados.rados.0.smithi117.stdout:356: expect (ObjNum 133 snap 33 seq_num 133) 2024-09-16T06:49:29.768 INFO:tasks.rados.rados.0.smithi117.stdout:357: setattr oid 14 current snap is 38 2024-09-16T06:49:29.768 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 1 to smithi11731613-42 2024-09-16T06:49:29.768 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 2 to smithi11731613-42 2024-09-16T06:49:29.768 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 3 to smithi11731613-42 2024-09-16T06:49:29.771 INFO:tasks.rados.rados.0.smithi117.stdout:358: snap_create 2024-09-16T06:49:29.776 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 4 to smithi11731613-42 2024-09-16T06:49:29.779 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.779 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.783 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:29.788+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:29.788 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 5 to smithi11731613-42 2024-09-16T06:49:29.788 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 6 to smithi11731613-42 2024-09-16T06:49:29.792 INFO:tasks.rados.rados.0.smithi117.stdout:355: finishing write tid 7 to smithi11731613-42 2024-09-16T06:49:29.792 INFO:tasks.rados.rados.0.smithi117.stdout:355: oid 42 updating version 0 to 86 2024-09-16T06:49:29.792 INFO:tasks.rados.rados.0.smithi117.stdout:355: oid 42 updating version 86 to 88 2024-09-16T06:49:29.792 INFO:tasks.rados.rados.0.smithi117.stdout:355: oid 42 version 88 is already newer than 87 2024-09-16T06:49:29.793 INFO:tasks.rados.rados.0.smithi117.stdout:355: oid 42 version 88 is already newer than 85 2024-09-16T06:49:29.793 INFO:tasks.rados.rados.0.smithi117.stdout:355: oid 42 updating version 88 to 89 2024-09-16T06:49:29.793 INFO:tasks.rados.rados.0.smithi117.stdout:355: oid 42 updating version 89 to 90 2024-09-16T06:49:29.793 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 42 v 90 (ObjNum 143 snap 38 seq_num 143) dirty exists 2024-09-16T06:49:29.793 INFO:tasks.rados.rados.0.smithi117.stdout:355: left oid 42 (ObjNum 143 snap 38 seq_num 143) 2024-09-16T06:49:29.794 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 14 v 153 (ObjNum 4113 snap 0 seq_num 1736782931) dirty exists 2024-09-16T06:49:29.794 INFO:tasks.rados.rados.0.smithi117.stdout:356: expect (ObjNum 133 snap 33 seq_num 133) 2024-09-16T06:49:29.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:49:29.885 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:29.885+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:29.917 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:29.917 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:29.932 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:49:30.001 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-16T06:49:30.002 INFO:teuthology.orchestra.run.smithi100.stdout:moved 4 / 130 (3.07692%) 2024-09-16T06:49:30.002 INFO:teuthology.orchestra.run.smithi100.stdout:avg 8.66667 2024-09-16T06:49:30.002 INFO:teuthology.orchestra.run.smithi100.stdout:stddev 1.73845 -> 1.61933 (expected baseline 2.8441) 2024-09-16T06:49:30.002 INFO:teuthology.orchestra.run.smithi100.stdout:min osd.4 with 6 -> 6 pgs (0.692308 -> 0.692308 * mean) 2024-09-16T06:49:30.002 INFO:teuthology.orchestra.run.smithi100.stdout:max osd.1 with 12 -> 12 pgs (1.38462 -> 1.38462 * mean) 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:oload 110 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:max_change 3 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:max_change_osds 4 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:average_utilization 0.0005 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:overload_utilization 0.0005 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 weight 1.0000 -> 0.6874 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13 weight 1.0000 -> 0.8168 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 weight 1.0000 -> 0.8829 2024-09-16T06:49:30.003 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 weight 1.0000 -> 0.9076 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:355: done (3 left) 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:356: done (2 left) 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:357: done (1 left) 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:358: done (0 left) 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:359: read oid 27 snap 33 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:359: expect (ObjNum 132 snap 33 seq_num 132) 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:360: read oid 6 snap -1 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:360: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:361: rollback oid 32 current snap is 39 2024-09-16T06:49:30.004 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 32 to 36 2024-09-16T06:49:30.005 INFO:tasks.rados.rados.0.smithi117.stdout:362: snap_create 2024-09-16T06:49:30.014 INFO:tasks.rados.rados.0.smithi117.stdout:361: finishing rollback tid 1 to smithi11731613-32 2024-09-16T06:49:30.014 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 32 v 137 (ObjNum 49 snap 0 seq_num 3825912384) dirty dne 2024-09-16T06:49:30.014 INFO:tasks.rados.rados.0.smithi117.stdout:360: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T06:49:30.021 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:30.021+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:30.031 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T06:49:30.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T06:49:30.086 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:30.082+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:30.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:49:30.187 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:30.186+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:30.279 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:30.279 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:30.289 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:30.286+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:30.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:49:30.390 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:30.390+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:30.415 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:30.415 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:30.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:49:30.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:30.491 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:30.489+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:30.552 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:30.553 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:30.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:49:30.590 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:30.590 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:30.592 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:30.595+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:30.614 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:30.637 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:30.637 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:30.664 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:30.664 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:30.690 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:30.690 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:30.690 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:30.694 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:30.693+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:30.697 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:30.697 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:30.697 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:30.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:49:30.714 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:30.715 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:30.741 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:30.741 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:30.766 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:30.767 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:30.794 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-09-16T06:49:30.794+0000 7f71e10b1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26544) UID: 0 2024-09-16T06:49:30.796 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:30.796 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:30.826 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:30.826 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:30.829 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:30.829 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:30.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:49:30.862 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:30.862 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:30.891 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:30.892 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:30.895 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:30.890+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:30.924 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:30.957 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:30.957 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:30.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:30.959+0000 7fb087fff640 0 -- 172.21.15.100:0/1968504850 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7fb09021ee00 msgr2=0x7fb090221210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=5, reqid=client.4511.0:59, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:30.967 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.024371704999993, 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063718670000000005, 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:30.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941153+0000", 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941187+0000", 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3837000000000001e-05 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002113+0000", 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060926368000000002 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002134+0000", 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-05 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002230+0000", 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5789000000000001e-05 2024-09-16T06:49:30.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002246+0000", 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5676e-05 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004821+0000", 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002575262 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004873+0000", 2024-09-16T06:49:30.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2095000000000002e-05 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=158, reqid=client.4511.0:77, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.989974666999998, 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051524863999999997, 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975550+0000", 2024-09-16T06:49:30.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975589+0000", 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9523000000000001e-05 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021558+0000", 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045968459000000003 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021574+0000", 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6399999999999999e-05 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021656+0000", 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1742000000000004e-05 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021673+0000", 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7343e-05 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027012+0000", 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053383889999999998 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027076+0000", 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4325999999999994e-05 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.9s1 35/32 ECSubRead(tid=11, to_read={3:92c79891:::smithi11731613-34:head=0,4096,0}, subchunks={3:92c79891:::smithi11731613-34:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.954163149999999, 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034091585000000001, 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011362+0000", 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011374+0000", 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2490000000000001e-05 2024-09-16T06:49:30.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044980+0000", 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033605818000000003 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044996+0000", 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5996999999999999e-05 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045095+0000", 2024-09-16T06:49:30.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9290000000000007e-05 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045454+0000", 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000359038 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.919934705000003, 2024-09-16T06:49:30.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623068000000002, 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045590+0000", 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045617+0000", 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.724e-05 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073137+0000", 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027519538999999999 2024-09-16T06:49:30.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073150+0000", 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3046e-05 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073218+0000", 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7879999999999994e-05 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:30.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073228+0000", 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074167+0000", 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093834699999999997 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074214+0000", 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7722000000000002e-05 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.0s3 35/32 ECSubRead(tid=19, to_read={3:043b7e8c:::smithi11731613-16:head=0,65536,0}, subchunks={3:043b7e8c:::smithi11731613-16:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.883780939000005, 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.14017474999999999, 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:30.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081744+0000", 2024-09-16T06:49:30.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081755+0000", 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0812000000000001e-05 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221448+0000", 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.13969311000000001 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221465+0000", 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7030000000000001e-05 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221629+0000", 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163549 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221920+0000", 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000291181 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.5s3 41/37 ECSubRead(tid=7, to_read={3:af79444c:::smithi11731613-29:head=0,118784,0}, subchunks={3:af79444c:::smithi11731613-29:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.001757132999998, 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29065317299999999, 2024-09-16T06:49:30.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963768+0000", 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963777+0000", 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2019999999999993e-06 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254192+0000", 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.290415064 2024-09-16T06:49:30.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254204+0000", 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1430000000000001e-05 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254309+0000", 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104763 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254422+0000", 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011342500000000001 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.108997914, 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017061956, 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856527+0000", 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856580+0000", 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2326e-05 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856691+0000", 2024-09-16T06:49:30.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011141600000000001 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856715+0000", 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3710000000000002e-05 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856826+0000", 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011109599999999999 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:30.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856852+0000", 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5686000000000001e-05 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873537+0000", 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016685114000000001 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873590+0000", 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3242000000000003e-05 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 59/32 ECSubWrite(tid=54, reqid=client.4511.0:469, at_version=59'36, trim_to=0'0, roll_forward_to=57'34))", 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.124928724, 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91193373600000005, 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:30.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840907+0000", 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000310045 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749945+0000", 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.90903863699999998 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749964+0000", 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8073000000000001e-05 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750059+0000", 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5636000000000004e-05 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750079+0000", 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9893000000000001e-05 2024-09-16T06:49:31.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752481+0000", 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024015989999999999 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752531+0000", 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0263000000000003e-05 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=90, reqid=osd.1.0:90, at_version=61'54, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.148601440999997, 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018170465, 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816986+0000", 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1818999999999999e-05 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817088+0000", 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102377 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817111+0000", 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2946e-05 2024-09-16T06:49:31.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817267+0000", 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015629899999999999 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817294+0000", 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6412999999999999e-05 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835044+0000", 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017750359 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835095+0000", 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1004999999999997e-05 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:31.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.145557500000002, 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175777999999999, 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819967+0000", 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820019+0000", 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2649999999999999e-05 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820184+0000", 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016481499999999999 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820207+0000", 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.245e-05 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820328+0000", 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120942 2024-09-16T06:49:31.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820352+0000", 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4732999999999999e-05 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835111+0000", 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014758378000000001 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835144+0000", 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3547000000000003e-05 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.133588564, 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016104653999999999, 2024-09-16T06:49:31.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831994+0000", 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7025999999999999e-05 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832116+0000", 2024-09-16T06:49:31.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012219299999999999 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832140+0000", 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4023e-05 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832493+0000", 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035320700000000002 2024-09-16T06:49:31.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832518+0000", 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5147000000000002e-05 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847982+0000", 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015463289 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848042+0000", 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0463e-05 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.es4 71/37 ECSubWrite(tid=22, reqid=osd.8.0:22, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.131094693000001, 2024-09-16T06:49:31.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019478348999999999, 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834430+0000", 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834474+0000", 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3739999999999998e-05 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834597+0000", 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122753 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834618+0000", 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1313000000000001e-05 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834923+0000", 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030448399999999999 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834948+0000", 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5293e-05 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853837+0000", 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018889153999999998 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853910+0000", 2024-09-16T06:49:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2368999999999998e-05 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=191, reqid=osd.11.0:191, at_version=71'82, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.129644007000003, 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175754, 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835882+0000", 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835880+0000", 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.835936+0000", 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6512999999999999e-05 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836143+0000", 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020638500000000001 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836164+0000", 2024-09-16T06:49:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1169999999999999e-05 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836644+0000", 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047988000000000002 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836664+0000", 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0616999999999999e-05 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.851001+0000", 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014336389999999999 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.851058+0000", 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6969000000000003e-05 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=144, reqid=client.4511.0:542, at_version=71'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.124580260999998, 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016044704, 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840944+0000", 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841000+0000", 2024-09-16T06:49:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5287000000000001e-05 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841092+0000", 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2009000000000006e-05 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841121+0000", 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.885e-05 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841388+0000", 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026696100000000002 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841425+0000", 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7626999999999997e-05 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856931+0000", 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015505755 2024-09-16T06:49:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856990+0000", 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9401999999999998e-05 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.124002167999997, 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016485414, 2024-09-16T06:49:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841522+0000", 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841612+0000", 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0575999999999995e-05 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841764+0000", 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015128200000000001 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841786+0000", 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1759999999999998e-05 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848074+0000", 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062882529999999997 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848099+0000", 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5510000000000001e-05 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857929+0000", 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098292880000000003 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858009+0000", 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0599000000000005e-05 2024-09-16T06:49:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=146, reqid=client.4511.0:543, at_version=71'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.123234500999999, 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018827678, 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842289+0000", 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842343+0000", 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842426+0000", 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3009000000000004e-05 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842448+0000", 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1860000000000001e-05 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850355+0000", 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079069680000000003 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850377+0000", 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2167e-05 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861052+0000", 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010674678 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861119+0000", 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7303000000000003e-05 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=147, reqid=client.4511.0:544, at_version=71'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.119598244000002, 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015229647000000001, 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845926+0000", 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845972+0000", 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5745999999999999e-05 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846062+0000", 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9909999999999998e-05 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846133+0000", 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1109e-05 2024-09-16T06:49:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853380+0000", 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072475899999999999 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853426+0000", 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5456e-05 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861126+0000", 2024-09-16T06:49:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077004730000000002 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861157+0000", 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1130000000000002e-05 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=132, reqid=osd.1.0:132, at_version=72'83, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.122191109999999, 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015387283999999999, 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843334+0000", 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844171+0000", 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083714800000000002 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845132+0000", 2024-09-16T06:49:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096062600000000001 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845482+0000", 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035013699999999999 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845634+0000", 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015229600000000001 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845661+0000", 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858652+0000", 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012991745000000001 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858722+0000", 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9822999999999999e-05 2024-09-16T06:49:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s6 72/37 ECSubWrite(tid=133, reqid=osd.1.0:133, at_version=72'36, trim_to=0'0, roll_forward_to=71'34))", 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.121341756, 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017512679999999999, 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844183+0000", 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844237+0000", 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4425999999999997e-05 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845784+0000", 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015470320000000001 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845809+0000", 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.455e-05 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845934+0000", 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012515900000000001 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845958+0000", 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3879999999999998e-05 2024-09-16T06:49:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861640+0000", 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01568172 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861697+0000", 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7173000000000003e-05 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=134, reqid=osd.1.0:134, at_version=72'85, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.120606958, 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017033627999999999, 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844918+0000", 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844980+0000", 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1920000000000003e-05 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846382+0000", 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014028630000000001 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846403+0000", 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0143e-05 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848953+0000", 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002550667 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848977+0000", 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3337e-05 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861911+0000", 2024-09-16T06:49:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012934783 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861953+0000", 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1192000000000001e-05 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.059 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:31.062 INFO:tasks.rados.rados.0.smithi117.stdout:360: done (3 left) 2024-09-16T06:49:31.062 INFO:tasks.rados.rados.0.smithi117.stdout:361: done (2 left) 2024-09-16T06:49:31.062 INFO:tasks.rados.rados.0.smithi117.stdout:362: done (1 left) 2024-09-16T06:49:31.062 INFO:tasks.rados.rados.0.smithi117.stdout:363: setattr oid 21 current snap is 40 2024-09-16T06:49:31.062 INFO:tasks.rados.rados.0.smithi117.stdout:364: snap_remove snap 17 2024-09-16T06:49:31.062 INFO:teuthology.orchestra.run.smithi100.stdout:moved 3 / 130 (2.30769%) 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:avg 8.66667 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:stddev 1.61933 -> 1.53478 (expected baseline 2.8441) 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:min osd.4 with 6 -> 6 pgs (0.692308 -> 0.692308 * mean) 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:max osd.1 with 12 -> 10 pgs (1.38462 -> 1.15385 * mean) 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:oload 110 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:max_change 0.05 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:max_change_osds 4 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:average_utilization 0.0005 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:overload_utilization 0.0005 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 weight 0.6874 -> 0.6374 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13 weight 0.8168 -> 0.7669 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 weight 0.8829 -> 0.8330 2024-09-16T06:49:31.063 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 weight 0.9076 -> 0.8576 2024-09-16T06:49:31.064 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:31.058+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:31.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:49:31.064 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T06:49:31.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-16T06:49:31.162 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:31.163+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:31.235 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:31.235 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:31.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:49:31.300 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:31.298+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:31.363 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:31.367+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:31.370 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:31.370 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:31.371 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:31.371 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:31.371 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:31.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:49:31.409 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:49:31.406+0000 7faa7cd62640 -1 mgr.server reply reply (4) Interrupted system call no change 2024-09-16T06:49:31.409 INFO:teuthology.orchestra.run.smithi100.stdout:moved 5 / 130 (3.84615%) 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:avg 8.66667 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:stddev 1.53478 -> 1.81965 (expected baseline 2.8441) 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:min osd.4 with 6 -> 7 pgs (0.692308 -> 0.807692 * mean) 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:max osd.14 with 11 -> 11 pgs (1.26923 -> 1.26923 * mean) 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:oload 110 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:max_change 3 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:max_change_osds 4 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:average_utilization 0.0005 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:overload_utilization 0.0005 2024-09-16T06:49:31.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 weight 0.6374 -> 0.4386 2024-09-16T06:49:31.411 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13 weight 0.7669 -> 0.6197 2024-09-16T06:49:31.411 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 weight 0.8330 -> 0.7362 2024-09-16T06:49:31.411 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 weight 0.8576 -> 0.7791 2024-09-16T06:49:31.411 INFO:teuthology.orchestra.run.smithi100.stderr:no change 2024-09-16T06:49:31.428 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T06:49:31.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T06:49:31.471 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:31.475+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.900196402000006, 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:49:31.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:49:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.086396223999998, 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:49:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:49:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.086267672000005, 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:49:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:49:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.086230345000004, 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.086095346999997, 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:49:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:49:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:49:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.086063186000004, 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.086034755, 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:49:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:49:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:49:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.085927835999996, 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:49:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:49:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.392421325000001, 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:49:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.392391076999999, 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:49:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:49:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.392361641999997, 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:49:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:49:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.392250723000004, 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:49:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:49:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:77 3.8s0 3:119b2380:::smithi11731613-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.579081196999994, 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055956466000000003, 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 77 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971691+0000", 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971701+0000", 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8519999999999999e-06 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971715+0000", 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4386000000000001e-05 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971725+0000", 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0360000000000001e-05 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.974948+0000", 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003222759 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975027+0000", 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8497000000000004e-05 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975520+0000", 2024-09-16T06:49:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049320500000000003 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.977378+0000", 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018576230000000001 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027491+0000", 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050112973999999998 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027649+0000", 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015816800000000001 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:78 3.8s0 3:119b2380:::smithi11731613-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.578986182999998, 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059293289999999998, 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971786+0000", 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971793+0000", 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-16T06:49:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971809+0000", 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5225000000000001e-05 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971820+0000", 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0922e-05 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975891+0000", 2024-09-16T06:49:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040709759999999996 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.976067+0000", 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017645900000000001 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027711+0000", 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051643840000000003 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027855+0000", 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014441399999999999 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031081+0000", 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032252359999999998 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 35/32 ECSubWriteReply(tid=162, last_complete=35'4, committed=1, applied=1))", 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.518020719999996, 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48350030599999999, 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032752+0000", 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032763+0000", 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1807000000000001e-05 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516055+0000", 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48329140799999998 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516068+0000", 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2782999999999999e-05 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516198+0000", 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129949 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516253+0000", 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5390000000000003e-05 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=17, reqid=client.4511.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.494924456000007, 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.51123769299999999, 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055848+0000", 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055863+0000", 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.464e-05 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565781+0000", 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50991880999999994 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565796+0000", 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4263e-05 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565932+0000", 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013609900000000001 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565949+0000", 2024-09-16T06:49:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7229000000000001e-05 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567019+0000", 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001070463 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567087+0000", 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7229000000000003e-05 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.7s2 40/37 ECSubRead(tid=11, to_read={3:e60b6e1b:::smithi11731613-14:head=0,4096,0}, subchunks={3:e60b6e1b:::smithi11731613-14:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 63.483243061000003, 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.375282063, 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067529+0000", 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067547+0000", 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8187000000000001e-05 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442125+0000", 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.37457846099999997 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442142+0000", 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6843000000000001e-05 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442313+0000", 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017038200000000001 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442812+0000", 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049983999999999998 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.709209815999998, 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030302045, 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841562+0000", 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841613+0000", 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1739e-05 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841692+0000", 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8974000000000007e-05 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841718+0000", 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5452999999999998e-05 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847063+0000", 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053447629999999998 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847087+0000", 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4553999999999999e-05 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871791+0000", 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024703433 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871866+0000", 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5133e-05 2024-09-16T06:49:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s4 71/32 ECSubWrite(tid=63, reqid=osd.12.0:63, at_version=71'48, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.707545936000002, 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623895, 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843226+0000", 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843302+0000", 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5922999999999998e-05 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843397+0000", 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5536000000000001e-05 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843419+0000", 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2240000000000001e-05 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847145+0000", 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037257950000000001 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847167+0000", 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1590000000000002e-05 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871800+0000", 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024633057 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871851+0000", 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1647000000000002e-05 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:564 3.8s0 3:1cef29f0:::smithi11731613-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.575290932000001, 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020010844, 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 564 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975481+0000", 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975500+0000", 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975518+0000", 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975531+0000", 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3207000000000001e-05 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.983862+0000", 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083305979999999998 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.984001+0000", 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000139629 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990097+0000", 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060955419999999998 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990219+0000", 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122349 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.995493+0000", 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052739930000000003 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:31.662 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:31.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:49:31.665 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:31.667+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:31.764 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:31.761+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:31.791 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:31.791 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:31.792 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:31.792 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:31.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:moved 1 / 130 (0.769231%) 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:avg 8.66667 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:stddev 1.53478 -> 1.61933 (expected baseline 2.8441) 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:min osd.4 with 6 -> 6 pgs (0.692308 -> 0.692308 * mean) 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:max osd.14 with 11 -> 11 pgs (1.26923 -> 1.26923 * mean) 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:oload 110 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:max_change 0.05 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:max_change_osds 4 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:average_utilization 0.0005 2024-09-16T06:49:31.828 INFO:teuthology.orchestra.run.smithi100.stdout:overload_utilization 0.0005 2024-09-16T06:49:31.829 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 weight 0.6374 -> 0.5874 2024-09-16T06:49:31.829 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13 weight 0.7669 -> 0.7169 2024-09-16T06:49:31.829 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 weight 0.8330 -> 0.7830 2024-09-16T06:49:31.829 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 weight 0.8576 -> 0.8076 2024-09-16T06:49:31.833 INFO:tasks.rados.rados.0.smithi117.stdout:364: done (2 left) 2024-09-16T06:49:31.833 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 1 current snap is 40 2024-09-16T06:49:31.833 INFO:tasks.rados.rados.0.smithi117.stdout:365: seq_num 144 ranges {2015232=16384} 2024-09-16T06:49:31.854 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T06:49:31.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-16T06:49:31.857 INFO:tasks.rados.rados.0.smithi117.stdout:365: writing smithi11731613-1 from 2015232 to 2031616 tid 1 2024-09-16T06:49:31.857 INFO:tasks.rados.rados.0.smithi117.stdout:366: copy_from oid 8 from oid 36 current snap is 40 2024-09-16T06:49:31.858 INFO:tasks.rados.rados.0.smithi117.stdout:367: delete oid 40 current snap is 40 2024-09-16T06:49:31.867 INFO:tasks.rados.rados.0.smithi117.stdout:367: done (4 left) 2024-09-16T06:49:31.867 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 28 current snap is 40 2024-09-16T06:49:31.867 INFO:tasks.rados.rados.0.smithi117.stdout:368: seq_num 146 ranges {0=49152} 2024-09-16T06:49:31.868 INFO:tasks.rados.rados.0.smithi117.stdout:368: writing smithi11731613-28 from 0 to 49152 tid 1 2024-09-16T06:49:31.868 INFO:tasks.rados.rados.0.smithi117.stdout:369: snap_remove snap 31 2024-09-16T06:49:31.869 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:31.865+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:31.940 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:31.940 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:31.941 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:31.941 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:31.941 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:31.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:49:31.965 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:31.967+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:32.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T06:49:32.066 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:32.065+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=2, reqid=client.4511.0:5, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.182687134000005, 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022146409999999998, 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897266+0000", 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897264+0000", 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897309+0000", 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5123000000000001e-05 2024-09-16T06:49:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913004+0000", 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015694784999999999 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913046+0000", 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2945999999999998e-05 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914353+0000", 2024-09-16T06:49:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001306104 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914377+0000", 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4562999999999998e-05 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919353+0000", 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049759360000000002 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919412+0000", 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9327000000000003e-05 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:22 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.180411590999995, 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023053033000000001, 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 22 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899542+0000", 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.899530+0000", 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901021+0000", 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014909019999999999 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901056+0000", 2024-09-16T06:49:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5136999999999999e-05 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901077+0000", 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1622999999999999e-05 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901274+0000", 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019618500000000001 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901606+0000", 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033268099999999998 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.903794+0000", 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021876339999999999 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907586+0000", 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003792489 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922327+0000", 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014741124 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922595+0000", 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026710500000000001 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:23 3.3s0 3:ce1274e9:::smithi11731613-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.178848388999995, 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022719541999999999, 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901105+0000", 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901103+0000", 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901115+0000", 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1722999999999999e-05 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901134+0000", 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9213000000000001e-05 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901149+0000", 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.523e-05 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905252+0000", 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041033170000000004 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905520+0000", 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026748400000000002 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.906744+0000", 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012242209999999999 2024-09-16T06:49:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923321+0000", 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016576822000000001 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923623+0000", 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249399999999999 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923824+0000", 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000200955 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:24 3.3s0 3:ce1274e9:::smithi11731613-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.178789562999995, 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033126111999999999, 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901162+0000", 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901169+0000", 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901187+0000", 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8320000000000001e-05 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901203+0000", 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5872999999999999e-05 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907294+0000", 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006090949 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907512+0000", 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021778499999999999 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924007+0000", 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016495668000000002 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924218+0000", 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021115199999999999 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934290+0000", 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010071302000000001 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:31 3.3s0 3:c7654c4c:::smithi11731613-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.178737240000004, 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026931531000000002, 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901215+0000", 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902241+0000", 2024-09-16T06:49:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010258190000000001 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902275+0000", 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4037e-05 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902312+0000", 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.718e-05 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053663829999999997 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.908030+0000", 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035200800000000001 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912004+0000", 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039736770000000001 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923909+0000", 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011905429 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927953+0000", 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040441499999999998 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928147+0000", 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194005 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:32 3.3s0 3:c7654c4c:::smithi11731613-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.177621181999996, 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028234555000000001, 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 32 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902330+0000", 2024-09-16T06:49:32.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902341+0000", 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1279e-05 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902360+0000", 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902374+0000", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3917000000000001e-05 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917230+0000", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014855607 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917486+0000", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025648799999999997 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918644+0000", 2024-09-16T06:49:32.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011572749999999999 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928363+0000", 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097195389999999993 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930283+0000", 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919505 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930566+0000", 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028369099999999998 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:33 3.3s0 3:c7654c4c:::smithi11731613-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.177566491999997, 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037765197, 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 33 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:32.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902386+0000", 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902392+0000", 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:49:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893999999999998e-05 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902429+0000", 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8689999999999999e-05 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934639+0000", 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032210404999999998 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934843+0000", 2024-09-16T06:49:32.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020376199999999999 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.940152+0000", 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053090200000000002 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=4, reqid=client.4511.0:44, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:32.120 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.162649806000005, 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788766000000001, 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917302+0000", 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917349+0000", 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7796000000000001e-05 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941504+0000", 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024154637 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941525+0000", 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0608999999999999e-05 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941640+0000", 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011531000000000001 2024-09-16T06:49:32.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941662+0000", 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2328999999999999e-05 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943041+0000", 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001379047 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943092+0000", 2024-09-16T06:49:32.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0834000000000002e-05 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:68 3.3s0 3:cbbd057c:::smithi11731613-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.125928192000003, 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045964098000000002, 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 68 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954024+0000", 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954031+0000", 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954046+0000", 2024-09-16T06:49:32.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954058+0000", 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1938e-05 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956544+0000", 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024857949999999998 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956742+0000", 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019819 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.957650+0000", 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090843 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959725+0000", 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074256 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999914+0000", 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040189895000000003 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999989+0000", 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4527e-05 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:69 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.125883707, 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048288992000000003, 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 69 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954068+0000", 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954074+0000", 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954088+0000", 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4866e-05 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954100+0000", 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1328999999999999e-05 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959103+0000", 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035749999999997 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959283+0000", 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017953899999999999 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000081+0000", 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040797893000000002 2024-09-16T06:49:32.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000261+0000", 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180304 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002358+0000", 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002097392 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:98 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.084459840999997, 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.086611296000000004, 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995492+0000", 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995501+0000", 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8720000000000008e-06 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995517+0000", 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6650000000000002e-05 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995529+0000", 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1447999999999999e-05 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.997874+0000", 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023456250000000001 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998037+0000", 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016212600000000001 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998796+0000", 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075897099999999999 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002377+0000", 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00358195 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081972+0000", 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.079594867 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082105+0000", 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013220899999999999 2024-09-16T06:49:32.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:99 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.084392777999994, 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.087893389000000002, 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995559+0000", 2024-09-16T06:49:32.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995565+0000", 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2659999999999998e-06 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995581+0000", 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.611e-05 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995594+0000", 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2269000000000001e-05 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999116+0000", 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003522238 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999227+0000", 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111483 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082346+0000", 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.083118743999999994 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082416+0000", 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9993000000000006e-05 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.083454+0000", 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00103751 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:158 3.3s0 3:c7654c4c:::smithi11731613-11:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.000166968000002, 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41982487699999999, 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079784+0000", 2024-09-16T06:49:32.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079794+0000", 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6260000000000007e-06 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079810+0000", 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6631000000000001e-05 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079821+0000", 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0849000000000001e-05 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079917+0000", 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5730999999999995e-05 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.080126+0000", 2024-09-16T06:49:32.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020895399999999999 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081069+0000", 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094364700000000004 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082604+0000", 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001534863 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499525+0000", 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41692029400000002 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499611+0000", 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-05 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:226 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:32.151 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.124286166999994, 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027980046000000001, 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 226 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955665+0000", 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955672+0000", 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000002e-06 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955690+0000", 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8179999999999999e-05 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955704+0000", 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.396e-05 2024-09-16T06:49:32.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956020+0000", 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000316101 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956038+0000", 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8425999999999999e-05 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974108+0000", 2024-09-16T06:49:32.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018069776999999999 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974338+0000", 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022928799999999999 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975586+0000", 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012484639999999999 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.979590+0000", 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040043939999999997 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983481+0000", 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038905480000000002 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983647+0000", 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166045 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:348 3.3s0 3:c742dae1:::smithi11731613-33:head [append 0~622592] snapc d=[d,c,b,9,8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.239263811000001, 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020152403999999999, 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 348 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840689+0000", 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.840687+0000", 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841715+0000", 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010285260000000001 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841743+0000", 2024-09-16T06:49:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7990000000000001e-05 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841758+0000", 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4686e-05 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842034+0000", 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027588800000000001 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.842198+0000", 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000164599 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.843642+0000", 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014432760000000001 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.847983+0000", 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043409349999999998 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860716+0000", 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012732902000000001 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.860842+0000", 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012617499999999999 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:351 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.237999787, 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023956603, 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 351 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841957+0000", 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0199999999999996e-06 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841966+0000", 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-06 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841974+0000", 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.34e-06 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849799+0000", 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078258089999999995 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.850139+0000", 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033906799999999999 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862313+0000", 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012174318999999999 2024-09-16T06:49:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862630+0000", 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031676099999999998 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.865910+0000", 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032802859999999999 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 47/32 ECSubWrite(tid=68, reqid=client.4511.0:361, at_version=0'0, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.208435694999999, 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50083384399999997, 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871516+0000", 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871541+0000", 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5782000000000002e-05 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370610+0000", 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.49906899100000002 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370625+0000", 2024-09-16T06:49:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.429e-05 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370744+0000", 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119746 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370773+0000", 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8807e-05 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372302+0000", 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015286519999999999 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372351+0000", 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9348999999999998e-05 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 49/32 ECSubWrite(tid=70, reqid=osd.3.0:70, at_version=49'51, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.224748523000002, 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023397583, 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855205+0000", 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855204+0000", 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967296 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.855221+0000", 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-16T06:49:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877682+0000", 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022460635999999999 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877690+0000", 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5369999999999997e-06 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877755+0000", 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4584000000000004e-05 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.877766+0000", 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0968000000000001e-05 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878577+0000", 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000811029 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:36.878602+0000", 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5191e-05 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 73/32 ECSubWriteReply(tid=108, last_complete=73'90, committed=1, applied=1))", 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.210193173, 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071784013999999993, 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.178 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869759+0000", 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869765+0000", 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7590000000000003e-06 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941447+0000", 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071682318999999994 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941458+0000", 2024-09-16T06:49:32.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1099e-05 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941509+0000", 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1201e-05 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941544+0000", 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4538e-05 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s5 94/77 ECSubWrite(tid=225, reqid=osd.1.0:225, at_version=94'164, trim_to=0'0, roll_forward_to=94'162))", 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:21.958805+0000", 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.121147946000001, 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020349823, 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.958805+0000", 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.958805+0000", 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.958804+0000", 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.958853+0000", 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9261999999999997e-05 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.975874+0000", 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017020315000000001 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.975892+0000", 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8671000000000001e-05 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.976009+0000", 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011705399999999999 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.976032+0000", 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2965000000000001e-05 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.979107+0000", 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030746800000000002 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:21.979155+0000", 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8009000000000001e-05 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.186 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:32.187 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:32.188 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:32.188 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:49:32.189 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:32.189 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:32.189 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:32.186+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:32.228 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-16T06:49:32.226+0000 7faa7cd62640 -1 mgr.server reply reply (3) No such process no change 2024-09-16T06:49:32.228 INFO:teuthology.orchestra.run.smithi100.stdout:moved 9 / 130 (6.92308%) 2024-09-16T06:49:32.228 INFO:teuthology.orchestra.run.smithi100.stdout:avg 8.66667 2024-09-16T06:49:32.228 INFO:teuthology.orchestra.run.smithi100.stdout:stddev 1.61933 -> 2.38979 (expected baseline 2.8441) 2024-09-16T06:49:32.228 INFO:teuthology.orchestra.run.smithi100.stdout:min osd.4 with 6 -> 7 pgs (0.692308 -> 0.807692 * mean) 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:max osd.14 with 11 -> 11 pgs (1.26923 -> 1.26923 * mean) 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:oload 110 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:max_change 1 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:max_change_osds 4 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:average_utilization 0.0005 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:overload_utilization 0.0006 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 weight 0.5874 -> 0.4089 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13 weight 0.7169 -> 0.5860 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 weight 0.7830 -> 0.7000 2024-09-16T06:49:32.229 INFO:teuthology.orchestra.run.smithi100.stderr:no change 2024-09-16T06:49:32.288 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:32.285+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:32.316 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:32.317 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:32.317 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:32.317 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:32.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:49:32.388 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:32.386+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:32.452 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:32.453 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:32.453 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:32.453 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:32.453 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:32.467 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:49:32.508 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:32.506+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:32.590 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:32.586+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:32.625 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:32.625 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:32.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:32.625 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:5 3.2s0 3:42d6f0b3:::smithi11731613-2:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.732723750999995, 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029483957000000002, 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892373+0000", 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892385+0000", 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1939999999999999e-05 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892403+0000", 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8202999999999998e-05 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892421+0000", 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8189000000000002e-05 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895208+0000", 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786527 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895392+0000", 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000184255 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896349+0000", 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00095644999999999997 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902998+0000", 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066495039999999997 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921651+0000", 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018652716 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921859+0000", 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208359 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:6 3.2s0 3:42d6f0b3:::smithi11731613-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.732642572000003, 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034217268000000002, 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892455+0000", 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892464+0000", 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.67e-06 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892484+0000", 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0265999999999999e-05 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892500+0000", 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.596e-05 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897287+0000", 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047868049999999999 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897527+0000", 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023998800000000001 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922181+0000", 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024653615 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922389+0000", 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020808600000000001 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926674+0000", 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042850639999999999 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:43 3.2s0 3:4636348c:::smithi11731613-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.719424519, 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020346564000000001, 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.637 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905671+0000", 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909244+0000", 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035731700000000001 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909296+0000", 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2352999999999998e-05 2024-09-16T06:49:32.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909321+0000", 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4813000000000001e-05 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909751+0000", 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042994099999999998 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.910155+0000", 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040433999999999999 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912510+0000", 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002354929 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924742+0000", 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012231967 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.925839+0000", 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001096302 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926021+0000", 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000182492 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:44 3.2s0 3:4636348c:::smithi11731613-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.642 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.715738888999994, 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034540078000000002, 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909357+0000", 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909370+0000", 2024-09-16T06:49:32.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2583e-05 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909390+0000", 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9930000000000001e-05 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909421+0000", 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1285999999999998e-05 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913550+0000", 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041288399999999999 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913768+0000", 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000217691 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914791+0000", 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010230770000000001 2024-09-16T06:49:32.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924746+0000", 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099553599999999999 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943736+0000", 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018990000999999999 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943900+0000", 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016405 2024-09-16T06:49:32.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:45 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.715663953000004, 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040625472000000003, 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.648 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909434+0000", 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909441+0000", 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0330000000000003e-06 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909460+0000", 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8674000000000001e-05 2024-09-16T06:49:32.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909473+0000", 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3485999999999999e-05 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919590+0000", 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010117482000000001 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919812+0000", 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000221389 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944680+0000", 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024868369000000001 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944880+0000", 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019949900000000001 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.950061+0000", 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051807770000000001 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:230 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.669502851999994, 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0201987, 2024-09-16T06:49:32.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 230 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955596+0000", 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955593+0000", 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955613+0000", 2024-09-16T06:49:32.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0630000000000001e-05 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955662+0000", 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8702999999999998e-05 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955689+0000", 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.73e-05 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955849+0000", 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015934200000000001 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955874+0000", 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.548e-05 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966673+0000", 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010799329999999999 2024-09-16T06:49:32.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.966857+0000", 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018410900000000001 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975795+0000", 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089374099999999998 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:231 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.669376342999996, 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021272859000000002, 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 231 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955721+0000", 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955733+0000", 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2129e-05 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955755+0000", 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1423000000000001e-05 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955770+0000", 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4790000000000001e-05 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955974+0000", 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020403199999999999 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955982+0000", 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6940000000000001e-06 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.967793+0000", 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011811056 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.968023+0000", 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023004500000000001 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976996+0000", 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089721929999999998 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:232 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.669151798000001, 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022137836000000001, 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 232 2024-09-16T06:49:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955946+0000", 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955958+0000", 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-05 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955977+0000", 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.912e-05 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955993+0000", 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5894000000000001e-05 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956101+0000", 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107402 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956123+0000", 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2056999999999999e-05 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971700+0000", 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015577155000000001 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971931+0000", 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023091400000000001 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.978085+0000", 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061543179999999998 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 55/37 ECSubWrite(tid=121, reqid=client.4511.0:431, at_version=55'56, trim_to=0'0, roll_forward_to=52'54))", 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.807536704999997, 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.102329752, 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817619+0000", 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7778999999999997e-05 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917448+0000", 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.099828728000000005 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917466+0000", 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7907e-05 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917613+0000", 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014708200000000001 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917643+0000", 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0315999999999999e-05 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919830+0000", 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021864940000000002 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919892+0000", 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2076000000000007e-05 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 71/37 ECSubWrite(tid=190, reqid=osd.11.0:190, at_version=71'82, trim_to=0'0, roll_forward_to=65'80))", 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.832354+0000", 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.792745236999998, 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017875147000000001, 2024-09-16T06:49:32.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832354+0000", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832354+0000", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832353+0000", 2024-09-16T06:49:32.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832404+0000", 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562999999999997e-05 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832509+0000", 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105066 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832535+0000", 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5772999999999999e-05 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.833293+0000", 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075863500000000002 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.833317+0000", 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-05 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850149+0000", 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016832041999999998 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850229+0000", 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9561999999999995e-05 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 71/37 ECSubWrite(tid=192, reqid=osd.11.0:192, at_version=71'84, trim_to=0'0, roll_forward_to=65'80))", 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.788420385999999, 2024-09-16T06:49:32.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022295459, 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836677+0000", 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836728+0000", 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1388999999999999e-05 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836823+0000", 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4826000000000003e-05 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836845+0000", 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2310000000000002e-05 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836965+0000", 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119603 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836989+0000", 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4258999999999999e-05 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857456+0000", 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020466868999999999 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858974+0000", 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015182590000000001 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s5 71/32 ECSubWrite(tid=62, reqid=osd.12.0:62, at_version=71'46, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.838936+0000", 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.786163066, 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020004181999999999, 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838936+0000", 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838936+0000", 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838935+0000", 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.838989+0000", 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3368999999999997e-05 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839096+0000", 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107113 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839119+0000", 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2887e-05 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839245+0000", 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012593500000000001 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.839269+0000", 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4493000000000001e-05 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858875+0000", 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019606068000000001 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858940+0000", 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4962999999999994e-05 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s1 71/37 ECSubWrite(tid=144, reqid=client.4511.0:542, at_version=71'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.784747649000003, 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018959849000000001, 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840351+0000", 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840405+0000", 2024-09-16T06:49:32.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4206000000000002e-05 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840522+0000", 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116945 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840544+0000", 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2350000000000001e-05 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841221+0000", 2024-09-16T06:49:32.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067698900000000002 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841246+0000", 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.497e-05 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.859269+0000", 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018022442999999999 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.859311+0000", 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2475999999999998e-05 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s5 71/32 ECSubWrite(tid=63, reqid=osd.12.0:63, at_version=71'48, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.843234+0000", 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.781865314000001, 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018047657000000002, 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843234+0000", 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843234+0000", 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843232+0000", 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843299+0000", 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7199000000000005e-05 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843410+0000", 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110893 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843432+0000", 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.19e-05 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850016+0000", 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065836469999999998 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850041+0000", 2024-09-16T06:49:32.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5179999999999999e-05 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861229+0000", 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011188600999999999 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861281+0000", 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1910000000000003e-05 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=71, reqid=client.4511.0:549, at_version=72'66, trim_to=0'0, roll_forward_to=72'64))", 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.727094876999999, 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051205101000000003, 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898003+0000", 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898034+0000", 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1466e-05 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948063+0000", 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050028524999999997 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948079+0000", 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5869999999999999e-05 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948191+0000", 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112663 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948206+0000", 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.484e-05 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.949167+0000", 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096042199999999995 2024-09-16T06:49:32.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.949209+0000", 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2706000000000001e-05 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=77, reqid=client.4511.0:555, at_version=72'72, trim_to=0'0, roll_forward_to=72'70))", 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.658994577000001, 2024-09-16T06:49:32.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036435799999999997, 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966102+0000", 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966153+0000", 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0362999999999999e-05 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000163+0000", 2024-09-16T06:49:32.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034010037999999999 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000187+0000", 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4159e-05 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000326+0000", 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013869500000000001 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000356+0000", 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0511999999999999e-05 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.002480+0000", 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021234090000000001 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.002540+0000", 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0733999999999998e-05 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=82, reqid=client.4511.0:572, at_version=72'77, trim_to=0'0, roll_forward_to=72'72))", 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.612871624, 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.301395462, 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012226+0000", 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012275+0000", 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8619999999999999e-05 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.302391+0000", 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29011663999999998 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.302400+0000", 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9409999999999996e-06 2024-09-16T06:49:32.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.305565+0000", 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031649389999999999 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.305627+0000", 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1673000000000004e-05 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.313590+0000", 2024-09-16T06:49:32.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079632660000000001 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.313623+0000", 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2855000000000003e-05 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 72/32 ECSubWriteReply(tid=114, last_complete=72'106, committed=1, applied=1))", 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:32.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.300450347999998, 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.17086979499999999, 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324647+0000", 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324660+0000", 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2982999999999999e-05 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495344+0000", 2024-09-16T06:49:32.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.17068462700000001 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495360+0000", 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5539000000000001e-05 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495453+0000", 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3367000000000002e-05 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495518+0000", 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5278999999999996e-05 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:806 3.2s0 3:476f74d7:::smithi11731613-27:head [append 0~671744] snapc 35=[33,31,2e,2b,2a,28,21,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.794663952000001, 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022399347, 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 806 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830432+0000", 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831511+0000", 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010790019999999999 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831552+0000", 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1624e-05 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831570+0000", 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8056000000000001e-05 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831718+0000", 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014737600000000001 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831744+0000", 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6622999999999999e-05 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836450+0000", 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047061350000000002 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836721+0000", 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027029199999999999 2024-09-16T06:49:32.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.838621+0000", 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019002820000000001 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.841994+0000", 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003373029 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.852763+0000", 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010769065 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.852834+0000", 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.127e-05 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:807 3.2s0 3:476f74d7:::smithi11731613-27:head [append 0~573440] snapc 35=[33,31,2e,2b,2a,28,21,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T06:49:32.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:16.836309+0000", 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.788789764000001, 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018722806000000002, 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 807 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836309+0000", 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836309+0000", 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836308+0000", 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:32.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836842+0000", 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000534436 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836856+0000", 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3905e-05 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836867+0000", 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0988999999999999e-05 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.840547+0000", 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003680114 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.840670+0000", 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012319699999999999 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.841393+0000", 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00072269499999999998 2024-09-16T06:49:32.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.843655+0000", 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022617819999999999 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.854946+0000", 2024-09-16T06:49:32.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011290826 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.855032+0000", 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6545999999999994e-05 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:32.724 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:32.726 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:49:32.726 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:32.726+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:32.827 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:32.831+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:32.828 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:369: done (5 left) 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:370: read oid 9 snap -1 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:370: expect deleted 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:371: copy_from oid 14 from oid 6 current snap is 40 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:365: finishing write tid 1 to smithi11731613-1 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:365: finishing write tid 2 to smithi11731613-1 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:372: copy_from oid 7 from oid 18 current snap is 40 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:365: finishing write tid 3 to smithi11731613-1 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:365: oid 1 updating version 0 to 93 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:365: oid 1 version 93 is already newer than 92 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 1 v 93 (ObjNum 144 snap 40 seq_num 144) dirty exists 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:365: left oid 1 (ObjNum 144 snap 40 seq_num 144) 2024-09-16T06:49:32.834 INFO:tasks.rados.rados.0.smithi117.stdout:368: finishing write tid 1 to smithi11731613-28 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:368: finishing write tid 2 to smithi11731613-28 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:366: finishing copy_from racing read to smithi11731613-8 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:366: finishing copy_from to smithi11731613-8 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 8 v 4 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:368: finishing write tid 3 to smithi11731613-28 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:368: oid 28 updating version 0 to 185 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:368: oid 28 version 185 is already newer than 184 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 28 v 185 (ObjNum 146 snap 40 seq_num 146) dirty exists 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:368: left oid 28 (ObjNum 146 snap 40 seq_num 146) 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:365: done (7 left) 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:366: done (6 left) 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:368: done (5 left) 2024-09-16T06:49:32.835 INFO:tasks.rados.rados.0.smithi117.stdout:373: delete oid 4 current snap is 40 2024-09-16T06:49:32.842 INFO:tasks.rados.rados.0.smithi117.stdout:373: done (5 left) 2024-09-16T06:49:32.842 INFO:tasks.rados.rados.0.smithi117.stdout:374: rollback oid 42 current snap is 40 2024-09-16T06:49:32.842 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 42 to 30 2024-09-16T06:49:32.842 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 3 current snap is 40 2024-09-16T06:49:32.843 INFO:tasks.rados.rados.0.smithi117.stdout:375: seq_num 148 ranges {1163264=81920} 2024-09-16T06:49:32.850 INFO:tasks.rados.rados.0.smithi117.stdout:375: writing smithi11731613-3 from 1163264 to 1245184 tid 1 2024-09-16T06:49:32.850 INFO:tasks.rados.rados.0.smithi117.stdout:376: setattr oid 26 current snap is 40 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:374: finishing rollback tid 0 to smithi11731613-42 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:374: finishing rollback tid 1 to smithi11731613-42 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:377: copy_from oid 30 from oid 37 current snap is 40 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:378: read oid 45 snap -1 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:378: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 35 current snap is 40 2024-09-16T06:49:32.855 INFO:tasks.rados.rados.0.smithi117.stdout:379: seq_num 149 ranges {1671168=98304} 2024-09-16T06:49:32.866 INFO:tasks.rados.rados.0.smithi117.stdout:379: writing smithi11731613-35 from 1671168 to 1769472 tid 1 2024-09-16T06:49:32.866 INFO:tasks.rados.rados.0.smithi117.stdout:374: finishing rollback tid 2 to smithi11731613-42 2024-09-16T06:49:32.866 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 42 v 97 (ObjNum 82 snap 11 seq_num 82) dirty exists 2024-09-16T06:49:32.866 INFO:tasks.rados.rados.0.smithi117.stdout:375: finishing write tid 1 to smithi11731613-3 2024-09-16T06:49:32.866 INFO:tasks.rados.rados.0.smithi117.stdout:375: finishing write tid 2 to smithi11731613-3 2024-09-16T06:49:32.866 INFO:tasks.rados.rados.0.smithi117.stdout:374: done (10 left) 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:380: rollback oid 39 current snap is 40 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 39 to 30 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:381: read oid 23 snap -1 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:381: expect (ObjNum 129 snap 32 seq_num 129) 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:382: rollback oid 1 current snap is 40 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 1 to 36 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 49 current snap is 40 2024-09-16T06:49:32.867 INFO:tasks.rados.rados.0.smithi117.stdout:383: seq_num 150 ranges {6340608=114688} 2024-09-16T06:49:32.877 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:961 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 94 in=55b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.016627746999999998, 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016660336000000001, 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.878 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 961 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861017+0000", 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861028+0000", 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1347e-05 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861059+0000", 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0338999999999999e-05 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861078+0000", 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.98e-05 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.862868+0000", 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017891809999999999 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863085+0000", 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000217359 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:962 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.016552348000000001, 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016671574000000002, 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "waiting for blocked object", 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 962 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861093+0000", 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861099+0000", 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6499999999999999e-06 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861122+0000", 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2629e-05 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861136+0000", 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.437e-05 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863286+0000", 2024-09-16T06:49:32.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021499359999999999 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863365+0000", 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8294000000000006e-05 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:966 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~98304 in=98304b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.867112+0000", 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.010534986, 2024-09-16T06:49:32.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010726552, 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 966 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867112+0000", 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867112+0000", 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867110+0000", 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867221+0000", 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011086099999999999 2024-09-16T06:49:32.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867239+0000", 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8017999999999999e-05 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867251+0000", 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2422000000000001e-05 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867402+0000", 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000150476 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867559+0000", 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015700499999999999 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868903+0000", 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013444590000000001 2024-09-16T06:49:32.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.872385+0000", 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034821409999999998 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:967 3.6s0 3:64026cd9:::smithi11731613-35:head [setxattr _header (57) in=64b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.867279+0000", 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.010367349, 2024-09-16T06:49:32.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010608898, 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 967 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867279+0000", 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867279+0000", 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867278+0000", 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867285+0000", 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2400000000000004e-06 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867295+0000", 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0328e-05 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867303+0000", 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8900000000000007e-06 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.870013+0000", 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027106280000000001 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.870261+0000", 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00024709899999999997 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.871003+0000", 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00074239100000000001 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.872561+0000", 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015579540000000001 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:968 3.6s0 3:64026cd9:::smithi11731613-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.867309+0000", 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.010337044, 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010637376, 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "waiting for rw locks", 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 968 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867309+0000", 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867309+0000", 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867309+0000", 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867313+0000", 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9319999999999998e-06 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867323+0000", 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8470000000000008e-06 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.867329+0000", 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.674e-06 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.871345+0000", 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0040159599999999998 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.871482+0000", 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013620700000000001 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.900 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:974 3.ds1 3:b273fc7d:::smithi11731613-1:head [append 0~0] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.868419+0000", 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.0092278159999999998, 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0096077569999999998, 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.901 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 974 2024-09-16T06:49:32.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.902 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.902 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868419+0000", 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868419+0000", 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868417+0000", 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868424+0000", 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9460000000000001e-06 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868439+0000", 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5062000000000001e-05 2024-09-16T06:49:32.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868450+0000", 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0412e-05 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868962+0000", 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051203900000000005 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:32.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.869232+0000", 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026958700000000001 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.871379+0000", 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021478880000000001 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.876020+0000", 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0046404189999999998 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:975 3.ds1 3:b273fc7d:::smithi11731613-1:head [rollback 40] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.868468+0000", 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.0091788449999999997, 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0096475099999999998, 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 975 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.908 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868468+0000", 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868468+0000", 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.909 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868467+0000", 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868471+0000", 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1479999999999999e-06 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868481+0000", 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.6209999999999999e-06 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868489+0000", 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7179999999999996e-06 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.871649+0000", 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031599969999999999 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.871751+0000", 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000102117 2024-09-16T06:49:32.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.872351+0000", 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00060013099999999995 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.876025+0000", 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0036739910000000001 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:976 3.ds1 3:b273fc7d:::smithi11731613-1:head [append 0~0] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.868495+0000", 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.0091517709999999995, 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.009709446, 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 976 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868495+0000", 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868495+0000", 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868494+0000", 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:32.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868499+0000", 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0089999999999998e-06 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868508+0000", 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.49e-06 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:32.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.868515+0000", 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8730000000000001e-06 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.873715+0000", 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052000340000000001 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.874018+0000", 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030364999999999998 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.875357+0000", 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013391639999999999 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.877806+0000", 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024488969999999998 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-16T06:49:32.917 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 8 2024-09-16T06:49:32.918 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:32.919 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:383: writing smithi11731613-49 from 6340608 to 6455296 tid 1 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:384: snap_create 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:371: finishing copy_from racing read to smithi11731613-14 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 26 v 45 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:375: finishing write tid 3 to smithi11731613-3 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:375: oid 3 updating version 0 to 90 2024-09-16T06:49:32.920 INFO:tasks.rados.rados.0.smithi117.stdout:375: oid 3 version 90 is already newer than 89 2024-09-16T06:49:32.921 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 3 v 90 (ObjNum 148 snap 40 seq_num 148) dirty exists 2024-09-16T06:49:32.921 INFO:tasks.rados.rados.0.smithi117.stdout:375: left oid 3 (ObjNum 148 snap 40 seq_num 148) 2024-09-16T06:49:32.921 INFO:tasks.rados.rados.0.smithi117.stdout:378: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-16T06:49:32.927 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:32.926+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:371: finishing copy_from to smithi11731613-14 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 14 v 19 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:379: finishing write tid 1 to smithi11731613-35 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:379: finishing write tid 2 to smithi11731613-35 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:382: finishing rollback tid 0 to smithi11731613-1 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:382: finishing rollback tid 1 to smithi11731613-1 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:382: finishing rollback tid 2 to smithi11731613-1 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 1 v 100 (ObjNum 119 snap 27 seq_num 119) dirty exists 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:379: finishing write tid 3 to smithi11731613-35 2024-09-16T06:49:32.941 INFO:tasks.rados.rados.0.smithi117.stdout:379: oid 35 updating version 0 to 38 2024-09-16T06:49:32.942 INFO:tasks.rados.rados.0.smithi117.stdout:379: oid 35 updating version 38 to 39 2024-09-16T06:49:32.942 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 35 v 39 (ObjNum 149 snap 40 seq_num 149) dirty exists 2024-09-16T06:49:32.942 INFO:tasks.rados.rados.0.smithi117.stdout:379: left oid 35 (ObjNum 149 snap 40 seq_num 149) 2024-09-16T06:49:32.942 INFO:tasks.rados.rados.0.smithi117.stdout:383: finishing write tid 1 to smithi11731613-49 2024-09-16T06:49:32.942 INFO:tasks.rados.rados.0.smithi117.stdout:381: expect (ObjNum 129 snap 32 seq_num 129) 2024-09-16T06:49:33.028 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:33.026+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:33.055 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:33.055 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:33.056 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:33.056 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:33.056 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:33.069 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:49:33.158 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:33.158+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:33.181 INFO:tasks.rados.rados.0.smithi117.stdout:377: finishing copy_from racing read to smithi11731613-30 2024-09-16T06:49:33.182 INFO:tasks.rados.rados.0.smithi117.stdout:377: finishing copy_from to smithi11731613-30 2024-09-16T06:49:33.182 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 30 v 94 (ObjNum 116 snap 26 seq_num 116) dirty exists 2024-09-16T06:49:33.230 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:33.229+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:75 3.as0 3:5d679340:::smithi11731613-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:33.237 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.272101508000006, 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033077179999999998, 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 75 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962304+0000", 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.239 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962308+0000", 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0600000000000001e-06 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962314+0000", 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.088e-06 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962319+0000", 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1120000000000004e-06 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.240 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964978+0000", 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026582540000000001 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965065+0000", 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7108999999999996e-05 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989270+0000", 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024205503 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989359+0000", 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9204999999999997e-05 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995382+0000", 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060223200000000003 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.242 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:95 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.242216932000005, 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042547695000000003, 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 95 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.243 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992188+0000", 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992194+0000", 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8760000000000002e-06 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992204+0000", 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.011e-05 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992212+0000", 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1890000000000001e-06 2024-09-16T06:49:33.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993832+0000", 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001620565 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993886+0000", 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4021999999999999e-05 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994477+0000", 2024-09-16T06:49:33.246 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059046499999999998 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995906+0000", 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014291919999999999 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034552+0000", 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038646106999999999 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.247 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034737+0000", 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018489900000000001 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:96 3.as0 3:5bc02ad9:::smithi11731613-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.242187948999998, 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.058410493000000001, 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.248 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 96 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992221+0000", 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2899999999999998e-06 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992229+0000", 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3080000000000008e-06 2024-09-16T06:49:33.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992236+0000", 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.528e-06 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994995+0000", 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027588700000000001 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:33.251 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995064+0000", 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9758000000000005e-05 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035940+0000", 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040876066000000003 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036021+0000", 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0243999999999999e-05 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.252 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050629+0000", 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014608009999999999 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=18, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.239106200999998, 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040520444000000003, 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.253 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.254 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995299+0000", 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995304+0000", 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5680000000000001e-06 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034371+0000", 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039067118999999997 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034380+0000", 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7949999999999996e-06 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034448+0000", 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8244000000000006e-05 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035820+0000", 2024-09-16T06:49:33.256 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001372306 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:144 3.as0 3:50fa89c4:::smithi11731613-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.170719731999995, 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38453364499999998, 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.257 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 144 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063691+0000", 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8879999999999996e-06 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063699+0000", 2024-09-16T06:49:33.259 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5799999999999992e-06 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063705+0000", 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.246e-06 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065212+0000", 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015062820000000001 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.260 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065312+0000", 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010052600000000001 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065706+0000", 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039373300000000001 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070934+0000", 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052282070000000003 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448115+0000", 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37718074400000001 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448220+0000", 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105104 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:145 3.as0 3:50fa89c4:::smithi11731613-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.170676763000003, 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38866241499999998, 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 145 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.263 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063732+0000", 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1999999999999999e-06 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063741+0000", 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4700000000000008e-06 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063748+0000", 2024-09-16T06:49:33.265 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4740000000000002e-06 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066083+0000", 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023351140000000001 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066155+0000", 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1650999999999996e-05 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.266 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448700+0000", 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38254526700000002 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448854+0000", 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154306 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.452392+0000", 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035374640000000001 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.267 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=23, last_complete=35'13, committed=1, applied=1))", 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.163411553000003, 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37764362699999998, 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070994+0000", 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.269 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.071000+0000", 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8980000000000001e-06 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447985+0000", 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37698559799999998 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447996+0000", 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0672999999999999e-05 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448044+0000", 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8338999999999999e-05 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448638+0000", 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059397499999999995 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.271 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:302 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 2] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.099543406000002, 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21732169600000001, 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 302 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.272 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134859+0000", 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134878+0000", 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9264999999999999e-05 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134922+0000", 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3260000000000003e-05 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134944+0000", 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2603e-05 2024-09-16T06:49:33.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135181+0000", 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023672400000000001 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135385+0000", 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020434300000000001 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.347993+0000", 2024-09-16T06:49:33.275 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21260742799999999 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.351735+0000", 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037421830000000001 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352007+0000", 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027182200000000001 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.276 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352184+0000", 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017777400000000001 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:303 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.099437236, 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21552337099999999, 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.277 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 303 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134967+0000", 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134976+0000", 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3860000000000007e-06 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.279 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134996+0000", 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0234999999999999e-05 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135015+0000", 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8949000000000001e-05 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135785+0000", 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000770388 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.280 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135925+0000", 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013959700000000001 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350066+0000", 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21414113400000001 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350271+0000", 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020513000000000001 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350492+0000", 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022106200000000001 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 61/32 ECSubWriteReply(tid=126, last_complete=61'99, committed=1, applied=1))", 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 44.358723597999997, 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63424799499999995, 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.282 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.283 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875681+0000", 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875692+0000", 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1154e-05 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508662+0000", 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63297013099999999 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508675+0000", 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3217000000000001e-05 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508759+0000", 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3652999999999993e-05 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.509930+0000", 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001171486 2024-09-16T06:49:33.285 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:542 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.405994905, 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031510007, 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 542 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.287 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828399+0000", 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828425+0000", 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5700000000000001e-05 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828469+0000", 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3510999999999997e-05 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828493+0000", 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4769999999999998e-05 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828815+0000", 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032114900000000001 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828827+0000", 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2143999999999999e-05 2024-09-16T06:49:33.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835549+0000", 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.006721856 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835824+0000", 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027592399999999998 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.838011+0000", 2024-09-16T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021865410000000002 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.849034+0000", 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011023141 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859755+0000", 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010721039999999999 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859921+0000", 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016604799999999999 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:543 3.6s0 3:64026cd9:::smithi11731613-35:head [rollback 24] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.405876669000001, 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034430898000000001, 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 543 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828528+0000", 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828538+0000", 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0457e-05 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828557+0000", 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8459000000000001e-05 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828571+0000", 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.447e-05 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828929+0000", 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035788200000000001 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.295 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828944+0000", 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4662e-05 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840136+0000", 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011191942 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840414+0000", 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027797100000000001 2024-09-16T06:49:33.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.842038+0000", 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0016248160000000001 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855171+0000", 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013132853 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.297 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862552+0000", 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073803990000000002 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862960+0000", 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00040862400000000001 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:544 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:33.298 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.405822100999998, 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034849585000000002, 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 544 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:33.299 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828583+0000", 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.300 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828589+0000", 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5819999999999998e-06 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828607+0000", 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7260000000000001e-05 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828620+0000", 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3789e-05 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828991+0000", 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037102199999999999 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828999+0000", 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5000000000000002e-06 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843716+0000", 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0147174 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843970+0000", 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000253732 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845224+0000", 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001253524 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855177+0000", 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0099530999999999994 2024-09-16T06:49:33.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863271+0000", 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0080939529999999992 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863434+0000", 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016288899999999999 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:743 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 34] snapc 2e=[2e,2b,2a,29,28,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 22.389229372999999, 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028808912999999998, 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 743 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845175+0000", 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:33.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845184+0000", 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9590000000000001e-06 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845198+0000", 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4282e-05 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845208+0000", 2024-09-16T06:49:33.307 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0258000000000001e-05 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845266+0000", 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7373000000000001e-05 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845364+0000", 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8196000000000006e-05 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.308 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.869612+0000", 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024247713000000001 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.871840+0000", 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022288270000000001 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.873933+0000", 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020920740000000002 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.873986+0000", 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3121000000000002e-05 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/32 ECSubWrite(tid=148, reqid=client.4511.0:775, at_version=0'0, trim_to=0'0, roll_forward_to=86'141))", 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.278910931999999, 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.83265717400000006, 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.310 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.311 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955494+0000", 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955505+0000", 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0801e-05 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786476+0000", 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.83097071499999997 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.312 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786491+0000", 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5109999999999999e-05 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786683+0000", 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000192165 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786704+0000", 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0873000000000002e-05 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.788074+0000", 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001370079 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.788152+0000", 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8248e-05 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.314 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:852 3.as0 3:57ffea71:::smithi11731613-30:head [append 0~0] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.297906566, 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.029643135000000001, 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 852 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.315 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936496+0000", 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:33.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936509+0000", 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.205e-05 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936550+0000", 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1384000000000002e-05 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936564+0000", 2024-09-16T06:49:33.317 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4053000000000001e-05 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936833+0000", 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026950699999999999 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936856+0000", 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2518999999999999e-05 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.318 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.941811+0000", 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00495547 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.941931+0000", 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011981699999999999 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943080+0000", 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001149129 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.319 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.965859+0000", 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022778732999999999 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966004+0000", 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014514199999999999 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966143+0000", 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000138391 2024-09-16T06:49:33.320 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:853 3.as0 3:57ffea71:::smithi11731613-30:head [rollback 15] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.29782084, 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.029774785000000002, 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.321 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 853 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936584+0000", 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936591+0000", 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.365e-06 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936603+0000", 2024-09-16T06:49:33.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2191999999999999e-05 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936612+0000", 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0350000000000007e-06 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936994+0000", 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038190700000000001 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937012+0000", 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8757e-05 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.947015+0000", 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010002341999999999 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.947373+0000", 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035835499999999998 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.950105+0000", 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027321899999999998 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966146+0000", 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016041161000000002 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966240+0000", 2024-09-16T06:49:33.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3199e-05 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966360+0000", 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012037 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:854 3.as0 3:57ffea71:::smithi11731613-30:head [append 0~0] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.297786613, 2024-09-16T06:49:33.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.030019711000000001, 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 854 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.329 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936623+0000", 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1659999999999996e-06 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936650+0000", 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7022e-05 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936658+0000", 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0809999999999994e-06 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.330 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937075+0000", 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000416628 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937091+0000", 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6586e-05 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.951259+0000", 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014167170999999999 2024-09-16T06:49:33.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.951679+0000", 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042059200000000001 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.952960+0000", 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012813690000000001 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966424+0000", 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013463298 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966523+0000", 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9567000000000004e-05 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966639+0000", 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000115837 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.333 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:961 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 94 in=55b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.37338731600000002, 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.092210185, 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 961 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.334 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861017+0000", 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861028+0000", 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1347e-05 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861059+0000", 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0338999999999999e-05 2024-09-16T06:49:33.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861078+0000", 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.98e-05 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.862868+0000", 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017891809999999999 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863085+0000", 2024-09-16T06:49:33.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000217359 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.940308+0000", 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.077223240999999998 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.943429+0000", 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031204499999999999 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.952785+0000", 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0093566599999999993 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.953229+0000", 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00044375099999999999 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:962 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:33.339 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 0.37331191699999999, 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.081297473999999995, 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 962 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:33.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861093+0000", 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.341 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861099+0000", 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6499999999999999e-06 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861122+0000", 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2629e-05 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861136+0000", 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.437e-05 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863286+0000", 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021499359999999999 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863365+0000", 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8294000000000006e-05 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.941742+0000", 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.078377418000000004 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.941953+0000", 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000210934 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.942392+0000", 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00043858299999999999 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.344 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.345 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:33.346 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:49:33.347 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:33.347+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:33.446 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:33.445+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:33.514 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:33.515 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:33.515 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:33.515 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:33.529 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:49:33.547 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:33.545+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:33.561 INFO:tasks.rados.rados.0.smithi117.stdout:383: finishing write tid 2 to smithi11731613-49 2024-09-16T06:49:33.565 INFO:tasks.rados.rados.0.smithi117.stdout:383: finishing write tid 3 to smithi11731613-49 2024-09-16T06:49:33.565 INFO:tasks.rados.rados.0.smithi117.stdout:383: oid 49 updating version 0 to 166 2024-09-16T06:49:33.566 INFO:tasks.rados.rados.0.smithi117.stdout:383: oid 49 version 166 is already newer than 165 2024-09-16T06:49:33.566 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 49 v 166 (ObjNum 150 snap 40 seq_num 150) dirty exists 2024-09-16T06:49:33.566 INFO:tasks.rados.rados.0.smithi117.stdout:383: left oid 49 (ObjNum 150 snap 40 seq_num 150) 2024-09-16T06:49:33.648 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:33.648 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:33.648 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:33.648 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:33.649 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:33.649 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:33.651+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:33.663 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:49:33.749 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:33.750+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:33.830 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:33.830 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:33.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:33.830 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:33.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.935566304999995, 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:49:33.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.935482508999996, 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:49:33.836 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:33.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:49:33.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.841 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.904077270000002, 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:49:33.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:49:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.847 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.904027502000005, 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.848 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:49:33.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:49:33.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.796289837000003, 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.853 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:33.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:49:33.858 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.795734378000006, 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.859 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.795678046999996, 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:49:33.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:49:33.869 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:219 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [copy-from ver 4] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 65.872458897000001, 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031057211000000001, 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.870 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 219 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956214+0000", 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956234+0000", 2024-09-16T06:49:33.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0275000000000002e-05 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956284+0000", 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0219999999999997e-05 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956309+0000", 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4930999999999999e-05 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956442+0000", 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013270999999999999 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956459+0000", 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7317999999999999e-05 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960744+0000", 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0042850090000000002 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961009+0000", 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026485300000000001 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.976584+0000", 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015575011999999999 2024-09-16T06:49:33.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.981658+0000", 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050738470000000003 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987101+0000", 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054435580000000003 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987274+0000", 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017305700000000001 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:220 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 65.872336438999994, 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023931746, 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.877 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 220 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956338+0000", 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956351+0000", 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2849e-05 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956375+0000", 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4195000000000001e-05 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956391+0000", 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5352000000000001e-05 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956650+0000", 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025920099999999997 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956669+0000", 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9157999999999999e-05 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961818+0000", 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051493709999999998 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961948+0000", 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012956 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979661+0000", 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017713421 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979887+0000", 2024-09-16T06:49:33.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022550399999999999 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.980271+0000", 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038454100000000002 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:316 3.ds0 3:b2c197ab:::smithi11731613-17:head [copy-from ver 4] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.666660936, 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017020408000000001, 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 316 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:33.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162012+0000", 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162030+0000", 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8688000000000001e-05 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162067+0000", 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6526000000000002e-05 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162083+0000", 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6061000000000001e-05 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162190+0000", 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107351 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162511+0000", 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032124399999999999 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.174275+0000", 2024-09-16T06:49:33.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011763691999999999 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.176204+0000", 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019286329999999999 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.178962+0000", 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027583099999999999 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.179036+0000", 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3456999999999998e-05 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:317 3.ds0 3:b2c197ab:::smithi11731613-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 63.666575600999998, 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013682241, 2024-09-16T06:49:33.889 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 317 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162106+0000", 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4620000000000001e-06 2024-09-16T06:49:33.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162124+0000", 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7706999999999998e-05 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162136+0000", 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2285e-05 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162962+0000", 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000826123 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163038+0000", 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5371999999999996e-05 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175269+0000", 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01223169 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175524+0000", 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025445300000000003 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175783+0000", 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025890899999999998 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.894 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:354 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.986138042999997, 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013614067000000001, 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 354 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842537+0000", 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:33.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842545+0000", 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2500000000000006e-06 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842562+0000", 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7512000000000001e-05 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842573+0000", 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0976e-05 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844555+0000", 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001981919 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844621+0000", 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5792999999999998e-05 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.856152+0000", 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011530883 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=96, reqid=osd.10.0:96, at_version=71'80, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.987941093000003, 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012334845000000001, 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.899 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840734+0000", 2024-09-16T06:49:33.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840800+0000", 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5324e-05 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840986+0000", 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000186701 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841011+0000", 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4372e-05 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841156+0000", 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000145749 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841215+0000", 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8845000000000003e-05 2024-09-16T06:49:33.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853010+0000", 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011794392000000001 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853070+0000", 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0158000000000002e-05 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=97, reqid=osd.10.0:97, at_version=71'82, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.987868581999997, 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013582399, 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840851+0000", 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4564999999999998e-05 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841065+0000", 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000213997 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841082+0000", 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6833e-05 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845856+0000", 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0047738549999999996 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845879+0000", 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854322+0000", 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084429220000000003 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854390+0000", 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7514000000000004e-05 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 71/37 ECSubWrite(tid=20, reqid=osd.5.0:20, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.985513892999997, 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013186145, 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843211+0000", 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9116000000000002e-05 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843322+0000", 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000111142 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843344+0000", 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2350000000000001e-05 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848942+0000", 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005598235 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848970+0000", 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7875999999999999e-05 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856277+0000", 2024-09-16T06:49:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073063049999999999 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856348+0000", 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1741000000000003e-05 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds0 72/54 ECSubWrite(tid=156, reqid=osd.3.0:156, at_version=72'61, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 33.990784980999997, 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013858534, 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837890+0000", 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837941+0000", 2024-09-16T06:49:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0958999999999998e-05 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838125+0000", 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000184014 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838144+0000", 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8258999999999999e-05 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838277+0000", 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000133452 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838303+0000", 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6038999999999999e-05 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851693+0000", 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013389803 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851750+0000", 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6641000000000002e-05 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 94/32 ECSubWrite(tid=260, reqid=client.4511.0:852, at_version=94'198, trim_to=0'0, roll_forward_to=92'196))", 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.885391873, 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014393538000000001, 2024-09-16T06:49:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943283+0000", 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943342+0000", 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8780999999999998e-05 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943490+0000", 2024-09-16T06:49:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014785699999999999 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943516+0000", 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5681999999999999e-05 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944924+0000", 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00140785 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944950+0000", 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6622e-05 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.957559+0000", 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012609120999999999 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.957678+0000", 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000118449 2024-09-16T06:49:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 94/77 ECSubWrite(tid=160, reqid=osd.13.0:160, at_version=94'29, trim_to=0'0, roll_forward_to=92'27))", 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.884494799, 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01233383, 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.924 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944271+0000", 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9946999999999998e-05 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944400+0000", 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012965399999999999 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944414+0000", 2024-09-16T06:49:33.925 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3454e-05 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944586+0000", 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000171893 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944603+0000", 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6923e-05 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.926 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956440+0000", 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011837640999999999 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956515+0000", 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4670000000000002e-05 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 94/32 ECSubWrite(tid=164, reqid=osd.10.0:164, at_version=94'136, trim_to=0'0, roll_forward_to=92'134))", 2024-09-16T06:49:33.927 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 11.884336187000001, 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012091431999999999, 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.928 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944367+0000", 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7620999999999999e-05 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.929 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944472+0000", 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105237 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944482+0000", 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7540000000000001e-06 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944702+0000", 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000220034 2024-09-16T06:49:33.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944716+0000", 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3465000000000001e-05 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956349+0000", 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011633596 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956431+0000", 2024-09-16T06:49:33.931 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2027000000000006e-05 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.6s5 104/102 ECSubWrite(tid=319, reqid=osd.3.0:319, at_version=104'34, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.9977447509999999, 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014382463, 2024-09-16T06:49:33.932 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.933 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830987+0000", 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6462000000000003e-05 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.934 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.831207+0000", 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021981799999999999 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.831229+0000", 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2103000000000002e-05 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.839034+0000", 2024-09-16T06:49:33.935 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0078046870000000003 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.839080+0000", 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6186000000000001e-05 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.845257+0000", 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0061766820000000002 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:33.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.845314+0000", 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6975000000000003e-05 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:33.938 INFO:tasks.rados.rados.0.smithi117.stdout:371: done (14 left) 2024-09-16T06:49:33.938 INFO:tasks.rados.rados.0.smithi117.stdout:375: done (13 left) 2024-09-16T06:49:33.938 INFO:tasks.rados.rados.0.smithi117.stdout:376: done (12 left) 2024-09-16T06:49:33.938 INFO:tasks.rados.rados.0.smithi117.stdout:377: done (11 left) 2024-09-16T06:49:33.938 INFO:tasks.rados.rados.0.smithi117.stdout:378: done (10 left) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:379: done (9 left) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:381: done (8 left) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:382: done (7 left) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:383: done (6 left) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:384: done (5 left) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:385: read oid 33 snap -1 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:385: expect deleted 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:386: read oid 37 snap 37 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:386: expect (ObjNum 116 snap 26 seq_num 116) 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 50 current snap is 41 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:387: seq_num 151 ranges {835584=131072} 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:387: writing smithi11731613-50 from 835584 to 966656 tid 1 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:388: rollback oid 2 current snap is 41 2024-09-16T06:49:33.939 INFO:tasks.rados.rados.0.smithi117.stdout:rollback oid 2 to 14 2024-09-16T06:49:33.940 INFO:tasks.rados.rados.0.smithi117.stdout:389: delete oid 42 current snap is 41 2024-09-16T06:49:33.940 INFO:tasks.rados.rados.0.smithi117.stdout:385: done (9 left) 2024-09-16T06:49:33.940 INFO:tasks.rados.rados.0.smithi117.stdout:389: done (8 left) 2024-09-16T06:49:33.940 INFO:tasks.rados.rados.0.smithi117.stdout:390: read oid 33 snap -1 2024-09-16T06:49:33.940 INFO:tasks.rados.rados.0.smithi117.stdout:390: expect deleted 2024-09-16T06:49:33.940 INFO:tasks.rados.rados.0.smithi117.stdout:391: snap_create 2024-09-16T06:49:33.941 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:49:33.941 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 21 v 201 (ObjNum 128 snap 32 seq_num 128) dirty exists 2024-09-16T06:49:33.942 INFO:tasks.rados.rados.0.smithi117.stdout:359: expect (ObjNum 132 snap 33 seq_num 132) 2024-09-16T06:49:33.943 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:33.941+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:33.968 INFO:tasks.rados.rados.0.smithi117.stdout:388: finishing rollback tid 0 to smithi11731613-2 2024-09-16T06:49:33.969 INFO:tasks.rados.rados.0.smithi117.stdout:386: expect (ObjNum 116 snap 26 seq_num 116) 2024-09-16T06:49:34.044 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:34.042+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:951 3.fs0 3:fa1c58b4:::smithi11731613-7:head [copy-from ver 32 in=55b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835109+0000", 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.2267333419999999, 2024-09-16T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.226790668, 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 951 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835109+0000", 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835109+0000", 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835108+0000", 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835111+0000", 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2310000000000001e-06 2024-09-16T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835120+0000", 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5569999999999996e-06 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835126+0000", 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5910000000000003e-06 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836529+0000", 2024-09-16T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001403219 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836530+0000", 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.511e-06 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864339+0000", 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0278086719999999 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864498+0000", 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015914999999999999 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.914456+0000", 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.049957930999999997 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.920830+0000", 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0063740689999999996 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-16T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 1 2024-09-16T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:34.075 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:49:34.145 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:34.147+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:34.190 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T06:49:34.231 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:34.232 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:34.232 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:34.232 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:34.232 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:34.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:49:34.246 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:34.246+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:34.347 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:34.351+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:34.358 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 35/32 ECSubWrite(tid=35, reqid=client.4511.0:152, at_version=35'10, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:49:34.361 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 71.280882481000006, 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.44133388099999998, 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077839+0000", 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077850+0000", 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0505e-05 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515878+0000", 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.43802848300000002 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515883+0000", 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1780000000000002e-06 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515923+0000", 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9866000000000001e-05 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515930+0000", 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.224e-06 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519096+0000", 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031664449999999999 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519174+0000", 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7590999999999996e-05 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:163 3.cs0 3:31c9f1c0:::smithi11731613-46:head [copy-from ver 2 in=54b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:34.366 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.846640790999999, 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.80778171600000004, 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 163 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512085+0000", 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8870000000000002e-06 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512091+0000", 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3539999999999996e-06 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512096+0000", 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8029999999999996e-06 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513808+0000", 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001712199 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513857+0000", 2024-09-16T06:49:34.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8649000000000001e-05 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.524232+0000", 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010375422 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.527189+0000", 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002956939 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319694+0000", 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79250512900000003 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319863+0000", 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168796 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.832013721999999, 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79400328899999995, 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526711+0000", 2024-09-16T06:49:34.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.18e-06 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319361+0000", 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79264922400000004 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319384+0000", 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3442e-05 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.375 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319508+0000", 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123683 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.320712+0000", 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012041 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:34.376 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.830038778000002, 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.394952786, 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528687+0000", 2024-09-16T06:49:34.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0810000000000004e-06 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923559+0000", 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39487182199999998 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923572+0000", 2024-09-16T06:49:34.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3406e-05 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923624+0000", 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1989e-05 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923636+0000", 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1990999999999999e-05 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.380 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=97, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 57.513922618000002, 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15568405900000001, 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.381 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844798+0000", 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844807+0000", 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8319999999999995e-06 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.998993+0000", 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15418552099999999 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999006+0000", 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3069e-05 2024-09-16T06:49:34.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999149+0000", 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000142965 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000484+0000", 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013350300000000001 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 49/37 ECSubWriteReply(tid=98, last_complete=49'26, committed=1, applied=1))", 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 57.508211696000004, 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15060995699999999, 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850509+0000", 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850523+0000", 2024-09-16T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4219000000000001e-05 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000937+0000", 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15041373399999999 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000960+0000", 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3768e-05 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001073+0000", 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112218 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001120+0000", 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7936999999999999e-05 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=99, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 57.356619680000001, 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123751468, 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.389 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002103+0000", 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002101+0000", 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.002109+0000", 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8399999999999995e-06 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125675+0000", 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.123566417 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125711+0000", 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5215000000000001e-05 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125821+0000", 2024-09-16T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110661 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.125854+0000", 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2638000000000002e-05 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:428 3.cs0 3:31c9f1c0:::smithi11731613-46:head [delete] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 53.503820431999998, 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.62638514099999998, 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 428 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854899+0000", 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.394 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854915+0000", 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5262999999999999e-05 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854948+0000", 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3005e-05 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854965+0000", 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7112000000000002e-05 2024-09-16T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855070+0000", 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105198 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855081+0000", 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1308000000000001e-05 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478008+0000", 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.622927122 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478196+0000", 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000187534 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.481287+0000", 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030910360000000001 2024-09-16T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(osd.5.22:1 3.fs0 3:fa1c58b4:::smithi11731613-7:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 53.490042191999997, 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.61158470300000001, 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868678+0000", 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868690+0000", 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2336e-05 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868722+0000", 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2122000000000002e-05 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868744+0000", 2024-09-16T06:49:34.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1855e-05 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868876+0000", 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013210300000000001 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868888+0000", 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2053e-05 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478066+0000", 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.60917769200000005 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478207+0000", 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000140743 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.480265+0000", 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020579219999999998 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.402 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:431 3.fs0 3:f3961c35:::smithi11731613-9:head [stat,delete] snapc 15=[15,11,e,d,c,b,8,5] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 51.543742057000003, 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.106237357, 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 431 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.403 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814980+0000", 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814977+0000", 2024-09-16T06:49:34.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.814993+0000", 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6095000000000001e-05 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815025+0000", 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1822000000000001e-05 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815042+0000", 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6803000000000002e-05 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815181+0000", 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013884600000000001 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.815482+0000", 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00030067499999999998 2024-09-16T06:49:34.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.817743+0000", 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022618769999999998 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.820149+0000", 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024051290000000002 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:34.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921014+0000", 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.10086516700000001 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:42.921218+0000", 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020381300000000001 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:583 3.fs0 3:fa1c58b4:::smithi11731613-7:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:34.408 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.299390729999999, 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41724843499999997, 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 583 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059336+0000", 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5140000000000001e-06 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059357+0000", 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0602e-05 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059374+0000", 2024-09-16T06:49:34.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7640000000000001e-05 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062783+0000", 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003409081 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062871+0000", 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7188999999999995e-05 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.063266+0000", 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039578200000000001 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.064116+0000", 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00084932599999999997 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476470+0000", 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41235417099999999 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476580+0000", 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011006999999999999 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.414 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 72/37 ECSubWriteReply(tid=209, last_complete=72'94, committed=1, applied=1))", 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.293162825000003, 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41187659199999999, 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065564+0000", 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2839999999999997e-06 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476221+0000", 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41065675400000001 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476236+0000", 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5509e-05 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476326+0000", 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9337000000000002e-05 2024-09-16T06:49:34.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.477436+0000", 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011104210000000001 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.1s1 81/77 ECSubWrite(tid=85, reqid=client.4511.0:732, at_version=0'0, trim_to=0'0, roll_forward_to=81'47))", 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 25.052098753999999, 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38681583800000002, 2024-09-16T06:49:34.418 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306621+0000", 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306649+0000", 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7733e-05 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692262+0000", 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.385613345 2024-09-16T06:49:34.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692278+0000", 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5642e-05 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692421+0000", 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014278699999999999 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692441+0000", 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0089e-05 2024-09-16T06:49:34.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693380+0000", 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00093900299999999999 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693439+0000", 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9342000000000002e-05 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.422 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDScrubReserve(3.as6 REQUEST e98)", 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:26.350367+0000", 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.0083554269999997, 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.42875901900000002, 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350367+0000", 2024-09-16T06:49:34.423 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350367+0000", 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350365+0000", 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.424 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350379+0000", 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3655999999999999e-05 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.778878+0000", 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.42849893700000002 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.778892+0000", 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.398e-05 2024-09-16T06:49:34.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.779028+0000", 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000136587 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.779057+0000", 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8615999999999998e-05 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.779126+0000", 2024-09-16T06:49:34.426 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8955000000000006e-05 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:934 3.fs0 3:fbae4168:::smithi11731613-21:head [create,setxattr tks`btPmsWciQOeYkTqWf`O^jQPan_cobnZerePO (1333) in=40b,setxattr \\]Q`aZUXRU_mQP_R_VijPtfgkatRsaNXgktfbort (102) in=40b,setxattr cnZX]a[_P_erokmkXY^dYWhTet_fSmmt`m\\nme`V (6244) in=40b,setxattr UkjTkbhYhVmTa[PPbrfXd\\VWd`QTYWbYamfUUmn_ (839) in=40b,setxattr nsi[hrP\\iPq\\jjZg^UtgbYkXj^dq\\fmSgdhbetoQ (1086) in=40b,setxattr [tgkYkY`fSah\\ahgj]RmkptX\\]h`T[gc]eag\\`b_ (2876) in=40b,setxattr QaOge_]eoPirtZlWSQ^\\gN_bgQOnfO]WdoqeracP (18032) in=40b,setxattr mi[nPcPgXbUbtkkjTY_pghrlZY_gk]Uk`_lmeTR\\ (1646) in=40b,setxattr oZtfSNa_W_re_`TbbYt[aPepifks`OqdQXYQa]od (2876) in=40b,setxattr QaYqal[djghoh`jsarPORqq_NqPekbXlacljdX\\f (350) in=40b,setxattr SiYahrpPZgdoSYbjtoXiggitTbphkZYhlW]ORlsa (523) in=40b,setxattr fPlNX^qbScs\\dcoWYSOmN_ZPPc_`rd\\TcQbjRod] (349) in=40b,setxattr PhkaTtafYaZ^nX]`bbR^QkYXepSSfjeqaYpgp]sa (12488) in=40b,setxattr aQ]`dlhjNXpgabS]eeS[j^Qhackqi`NZap[nUla\\ (6736) in=40b,setxattr QbfiUncStonfq^rXh`RaOPPmaOtpUpPX_]ob_\\\\t (6244) in=40b,setxattr UPURgp\\idZifafiaW[dX^WQSa_mZ]lsORaprWQYn (875) in=40b,setxattr N]e]t\\kgiWZilkcVqpgg (208) in=20b,setxattr YknjWPR]WkZl[e\\OiP\\[QYNb_RjhrsddS[RiOW[a (14664) in=40b,setxattr iWnNoigT`Pa_fY[[dPqrrPaTl_ptOPchg\\]theNm (1683) in=40b,setxattr RPrnabOkbfqbSYdbXaTf_gh`S^RgP^ohg`Rddbqn (805) in=40b] snapc 44=[44,43,41,40,3e,3c,3a,33,31,28,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:31.023596+0000", 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 3.3351263919999998, 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8449128350000001, 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.427 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 934 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023596+0000", 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023596+0000", 2024-09-16T06:49:34.428 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023593+0000", 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023747+0000", 2024-09-16T06:49:34.429 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154363 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023786+0000", 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8636000000000001e-05 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023810+0000", 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4006e-05 2024-09-16T06:49:34.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023897+0000", 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7430999999999997e-05 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023903+0000", 2024-09-16T06:49:34.431 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3550000000000004e-06 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836210+0000", 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.81230698 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836213+0000", 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3299999999999999e-06 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.432 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.858868+0000", 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.022654876 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.859317+0000", 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000449016 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.861268+0000", 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001951604 2024-09-16T06:49:34.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.863681+0000", 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024128109999999999 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.868047+0000", 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.004366188 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.434 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.868509+0000", 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00046132200000000001 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 104/77 ECSubWrite(tid=226, reqid=osd.13.0:226, at_version=104'159, trim_to=0'0, roll_forward_to=104'157))", 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:31.869610+0000", 2024-09-16T06:49:34.435 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 2.4891124979999999, 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.70657596600000006, 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869610+0000", 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.436 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869610+0000", 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869608+0000", 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869656+0000", 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7533e-05 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573471+0000", 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.70381538499999996 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573487+0000", 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5498999999999998e-05 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573632+0000", 2024-09-16T06:49:34.438 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014505 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573652+0000", 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0440000000000001e-05 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.576125+0000", 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002472756 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.439 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.576186+0000", 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0649000000000002e-05 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:946 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~0,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835018+0000", 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.5237039000000001, 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.032809079, 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.440 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 946 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835018+0000", 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.441 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835018+0000", 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835016+0000", 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835024+0000", 2024-09-16T06:49:34.442 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835049+0000", 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4983000000000001e-05 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835059+0000", 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.059e-05 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.443 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836243+0000", 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011832559999999999 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836243+0000", 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8500000000000001e-07 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.863780+0000", 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0275364920000001 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.444 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for missing object", 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.863953+0000", 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017281100000000001 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.867709+0000", 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037559519999999999 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.445 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.867827+0000", 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011885200000000001 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:947 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~0,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.5236537509999999, 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0342719360000001, 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.446 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 947 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.447 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835072+0000", 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1180000000000002e-06 2024-09-16T06:49:34.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835079+0000", 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4330000000000001e-06 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835084+0000", 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9969999999999998e-06 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.449 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836346+0000", 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001262958 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836348+0000", 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.468e-06 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864122+0000", 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.027773923 2024-09-16T06:49:34.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for missing object", 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864170+0000", 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8010999999999998e-05 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869237+0000", 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005067111 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869340+0000", 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000103415 2024-09-16T06:49:34.451 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:948 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~0,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835089+0000", 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.5236336449999999, 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0344568949999999, 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.452 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 948 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835089+0000", 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835089+0000", 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.453 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835088+0000", 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835091+0000", 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0350000000000002e-06 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835098+0000", 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2520000000000002e-06 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.454 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835104+0000", 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4029999999999998e-06 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836442+0000", 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013385960000000001 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836447+0000", 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8930000000000004e-06 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.455 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864211+0000", 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0277632809999999 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for missing object", 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864301+0000", 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0840000000000004e-05 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869379+0000", 2024-09-16T06:49:34.456 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050770750000000003 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869545+0000", 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000166921 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:952 3.fs0 3:fa1c58b4:::smithi11731613-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835130+0000", 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 1.523592407, 2024-09-16T06:49:34.457 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.084286708, 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 952 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835130+0000", 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.458 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835130+0000", 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835129+0000", 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835134+0000", 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5759999999999999e-06 2024-09-16T06:49:34.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835140+0000", 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4849999999999997e-06 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835147+0000", 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6370000000000002e-06 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836576+0000", 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014293450000000001 2024-09-16T06:49:34.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836579+0000", 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1210000000000001e-06 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864677+0000", 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.028098347 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:34.461 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864808+0000", 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013064399999999999 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.918901+0000", 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.054093585999999999 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.919074+0000", 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017242200000000001 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.919417+0000", 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00034298500000000002 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.463 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:34.464 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:34.465 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:34.465 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:34.466 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:34.466 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:49:34.466 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:34.467 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:34.467 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:34.467+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:34.509 INFO:tasks.rados.rados.0.smithi117.stdout:387: finishing write tid 1 to smithi11731613-50 2024-09-16T06:49:34.509 INFO:tasks.rados.rados.0.smithi117.stdout:387: finishing write tid 2 to smithi11731613-50 2024-09-16T06:49:34.509 INFO:tasks.rados.rados.0.smithi117.stdout:387: finishing write tid 3 to smithi11731613-50 2024-09-16T06:49:34.509 INFO:tasks.rados.rados.0.smithi117.stdout:387: oid 50 updating version 0 to 168 2024-09-16T06:49:34.509 INFO:tasks.rados.rados.0.smithi117.stdout:387: oid 50 updating version 168 to 169 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 50 v 169 (ObjNum 151 snap 41 seq_num 151) dirty exists 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:387: left oid 50 (ObjNum 151 snap 41 seq_num 151) 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:372: finishing copy_from racing read to smithi11731613-7 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:380: finishing rollback tid 1 to smithi11731613-39 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:388: finishing rollback tid 1 to smithi11731613-2 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:388: finishing rollback tid 2 to smithi11731613-2 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 2 v 200 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:380: finishing rollback tid 2 to smithi11731613-39 2024-09-16T06:49:34.510 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 39 v 201 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-09-16T06:49:34.566 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:34.565+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:34.588 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:34.589 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:34.589 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:34.589 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:34.602 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:49:34.690 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:34.690+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:34.759 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:34.759 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:34.759 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:34.768 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:34.771+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:34.771 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:49:34.834 INFO:tasks.rados.rados.0.smithi117.stdout:359: done (9 left) 2024-09-16T06:49:34.834 INFO:tasks.rados.rados.0.smithi117.stdout:363: done (8 left) 2024-09-16T06:49:34.834 INFO:tasks.rados.rados.0.smithi117.stdout:370: done (7 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:380: done (6 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:386: done (5 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:387: done (4 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:388: done (3 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:390: done (2 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:391: done (1 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:392: rmattr oid 49 current snap is 42 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:392: done (1 left) 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 6 current snap is 42 2024-09-16T06:49:34.835 INFO:tasks.rados.rados.0.smithi117.stdout:393: seq_num 153 ranges {32768=163840} 2024-09-16T06:49:34.836 INFO:tasks.rados.rados.0.smithi117.stdout:393: writing smithi11731613-6 from 32768 to 196608 tid 1 2024-09-16T06:49:34.841 INFO:tasks.rados.rados.0.smithi117.stdout:394: copy_from oid 38 from oid 36 current snap is 42 2024-09-16T06:49:34.841 INFO:tasks.rados.rados.0.smithi117.stdout:395: read oid 9 snap -1 2024-09-16T06:49:34.841 INFO:tasks.rados.rados.0.smithi117.stdout:395: expect deleted 2024-09-16T06:49:34.841 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 5 current snap is 42 2024-09-16T06:49:34.842 INFO:tasks.rados.rados.0.smithi117.stdout:396: seq_num 154 ranges {81920=180224} 2024-09-16T06:49:34.842 INFO:tasks.rados.rados.0.smithi117.stdout:396: writing smithi11731613-5 from 81920 to 262144 tid 1 2024-09-16T06:49:34.842 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 20 current snap is 42 2024-09-16T06:49:34.842 INFO:tasks.rados.rados.0.smithi117.stdout:397: seq_num 155 ranges {2097152=212992} 2024-09-16T06:49:34.855 INFO:tasks.rados.rados.0.smithi117.stdout:397: writing smithi11731613-20 from 2097152 to 2310144 tid 1 2024-09-16T06:49:34.855 INFO:tasks.rados.rados.0.smithi117.stdout:append oid 15 current snap is 42 2024-09-16T06:49:34.855 INFO:tasks.rados.rados.0.smithi117.stdout:398: seq_num 156 ranges {32768=229376} 2024-09-16T06:49:34.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:49:34.858 INFO:tasks.rados.rados.0.smithi117.stdout:398: writing smithi11731613-15 from 32768 to 262144 tid 1 2024-09-16T06:49:34.858 INFO:tasks.rados.rados.0.smithi117.stdout:399: delete oid 41 current snap is 42 2024-09-16T06:49:34.858 INFO:tasks.rados.rados.0.smithi117.stdout:372: finishing copy_from to smithi11731613-7 2024-09-16T06:49:34.858 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 7 v 32 (ObjNum 115 snap 26 seq_num 115) dirty exists 2024-09-16T06:49:34.858 INFO:tasks.rados.rados.0.smithi117.stdout:393: finishing write tid 1 to smithi11731613-6 2024-09-16T06:49:34.858 INFO:tasks.rados.rados.0.smithi117.stdout:393: finishing write tid 2 to smithi11731613-6 2024-09-16T06:49:34.859 INFO:tasks.rados.rados.0.smithi117.stdout:394: finishing copy_from racing read to smithi11731613-38 2024-09-16T06:49:34.859 INFO:tasks.rados.rados.0.smithi117.stdout:396: finishing write tid 1 to smithi11731613-5 2024-09-16T06:49:34.862 INFO:tasks.rados.rados.0.smithi117.stdout:394: finishing copy_from to smithi11731613-38 2024-09-16T06:49:34.862 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 38 v 4 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-16T06:49:34.865 INFO:tasks.rados.rados.0.smithi117.stdout:397: finishing write tid 1 to smithi11731613-20 2024-09-16T06:49:34.865 INFO:tasks.rados.rados.0.smithi117.stdout:393: finishing write tid 3 to smithi11731613-6 2024-09-16T06:49:34.865 INFO:tasks.rados.rados.0.smithi117.stdout:393: oid 6 updating version 0 to 39 2024-09-16T06:49:34.865 INFO:tasks.rados.rados.0.smithi117.stdout:393: oid 6 updating version 39 to 40 2024-09-16T06:49:34.865 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 6 v 40 (ObjNum 153 snap 42 seq_num 153) dirty exists 2024-09-16T06:49:34.865 INFO:tasks.rados.rados.0.smithi117.stdout:393: left oid 6 (ObjNum 153 snap 42 seq_num 153) 2024-09-16T06:49:34.866 INFO:tasks.rados.rados.0.smithi117.stdout:397: finishing write tid 2 to smithi11731613-20 2024-09-16T06:49:34.868 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:34.865+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:34.870 INFO:tasks.rados.rados.0.smithi117.stdout:397: finishing write tid 3 to smithi11731613-20 2024-09-16T06:49:34.871 INFO:tasks.rados.rados.0.smithi117.stdout:397: oid 20 updating version 0 to 67 2024-09-16T06:49:34.871 INFO:tasks.rados.rados.0.smithi117.stdout:397: oid 20 version 67 is already newer than 66 2024-09-16T06:49:34.871 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 20 v 67 (ObjNum 155 snap 42 seq_num 155) dirty exists 2024-09-16T06:49:34.871 INFO:tasks.rados.rados.0.smithi117.stdout:397: left oid 20 (ObjNum 155 snap 42 seq_num 155) 2024-09-16T06:49:34.907 INFO:tasks.rados.rados.0.smithi117.stdout:396: finishing write tid 2 to smithi11731613-5 2024-09-16T06:49:34.909 INFO:tasks.rados.rados.0.smithi117.stdout:396: finishing write tid 3 to smithi11731613-5 2024-09-16T06:49:34.909 INFO:tasks.rados.rados.0.smithi117.stdout:396: oid 5 updating version 0 to 203 2024-09-16T06:49:34.909 INFO:tasks.rados.rados.0.smithi117.stdout:396: oid 5 version 203 is already newer than 202 2024-09-16T06:49:34.909 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 5 v 203 (ObjNum 154 snap 42 seq_num 154) dirty exists 2024-09-16T06:49:34.909 INFO:tasks.rados.rados.0.smithi117.stdout:396: left oid 5 (ObjNum 154 snap 42 seq_num 154) 2024-09-16T06:49:34.950 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:34.950 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:266 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 65.995925554999999, 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014774862999999999, 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.951 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 266 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:34.952 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953825+0000", 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953846+0000", 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0171999999999999e-05 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953889+0000", 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3284000000000001e-05 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953913+0000", 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4071e-05 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.954 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954055+0000", 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014240699999999999 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954076+0000", 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0678000000000001e-05 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961148+0000", 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070716959999999997 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.955 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961402+0000", 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025470100000000003 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.968604+0000", 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072013340000000002 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.956 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:267 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 65.995809343000005, 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015485804000000001, 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 267 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953944+0000", 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.958 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953973+0000", 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9581e-05 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953995+0000", 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1685e-05 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954011+0000", 2024-09-16T06:49:34.959 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5862000000000002e-05 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954223+0000", 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021187199999999999 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954233+0000", 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0789e-05 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.960 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962540+0000", 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0083068819999999998 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962710+0000", 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017007799999999999 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.969431+0000", 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0067204250000000004 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.961 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:268 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 65.995729096000005, 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.30266284300000001, 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:34.962 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 268 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.963 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954024+0000", 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954032+0000", 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4290000000000004e-06 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954050+0000", 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8332e-05 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.964 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954063+0000", 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2955999999999999e-05 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954379+0000", 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031616000000000001 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954389+0000", 2024-09-16T06:49:34.965 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3959999999999995e-06 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963428+0000", 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0090396959999999998 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963723+0000", 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00029431800000000001 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.966 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:29.256688+0000", 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.29296564200000003 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 58.114466907000001, 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014247134, 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.967 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.968 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835286+0000", 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835349+0000", 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3258999999999999e-05 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835459+0000", 2024-09-16T06:49:34.969 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110097 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835480+0000", 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1424999999999999e-05 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835600+0000", 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011922 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835623+0000", 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3144999999999999e-05 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849454+0000", 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013830779 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849535+0000", 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1021999999999998e-05 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 61/32 ECSubWrite(tid=83, reqid=osd.14.0:83, at_version=61'72, trim_to=0'0, roll_forward_to=59'68))", 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 46.116181730000001, 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012777472999999999, 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833571+0000", 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833631+0000", 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9917999999999998e-05 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833761+0000", 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013055799999999999 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833784+0000", 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2714999999999999e-05 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.974 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835926+0000", 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021422720000000002 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835943+0000", 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6889e-05 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846281+0000", 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010337641999999999 2024-09-16T06:49:34.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846350+0000", 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9315999999999998e-05 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 61/32 ECSubWrite(tid=85, reqid=osd.10.0:85, at_version=61'70, trim_to=0'0, roll_forward_to=59'66))", 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 46.111299997000003, 2024-09-16T06:49:34.976 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013138069, 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838452+0000", 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838506+0000", 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3594999999999998e-05 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839270+0000", 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00076359400000000003 2024-09-16T06:49:34.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839293+0000", 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3006e-05 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839420+0000", 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127355 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839446+0000", 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851525+0000", 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012079196 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851592+0000", 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7156999999999994e-05 2024-09-16T06:49:34.980 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=101, reqid=osd.10.0:101, at_version=72'86, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.112286803000003, 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017459348, 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837467+0000", 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.982 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837538+0000", 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0541000000000001e-05 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837664+0000", 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012650899999999999 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837687+0000", 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2759999999999999e-05 2024-09-16T06:49:34.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839677+0000", 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019901300000000001 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839707+0000", 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0049e-05 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.984 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854862+0000", 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015155366 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854927+0000", 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4629999999999996e-05 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.985 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=157, reqid=osd.3.0:157, at_version=72'123, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.111973849999998, 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027234286, 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.986 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837780+0000", 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837839+0000", 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8801000000000002e-05 2024-09-16T06:49:34.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837959+0000", 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000120436 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837982+0000", 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3156e-05 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838118+0000", 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000135673 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838143+0000", 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4899e-05 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.864948+0000", 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026805539999999999 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865015+0000", 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6501000000000006e-05 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=102, reqid=osd.10.0:102, at_version=72'88, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.109162865999998, 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018351104, 2024-09-16T06:49:34.990 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840590+0000", 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840640+0000", 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0631999999999999e-05 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840728+0000", 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7933e-05 2024-09-16T06:49:34.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840750+0000", 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2022999999999999e-05 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840864+0000", 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113423 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840889+0000", 2024-09-16T06:49:34.993 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5092999999999998e-05 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858883+0000", 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017994231999999999 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858943+0000", 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9587999999999999e-05 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=158, reqid=osd.3.0:158, at_version=72'125, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.107521728999998, 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022836532999999999, 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842231+0000", 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:34.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842295+0000", 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4238000000000004e-05 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842418+0000", 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012244299999999999 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:34.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842441+0000", 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3606000000000001e-05 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850726+0000", 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082850219999999995 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850750+0000", 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3822e-05 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865034+0000", 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014283563000000001 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865069+0000", 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5712e-05 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:34.999 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=65, reqid=osd.12.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.104198703999998, 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017333787, 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845555+0000", 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845609+0000", 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4284999999999999e-05 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.002 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845766+0000", 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015658100000000001 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845789+0000", 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2866000000000001e-05 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845942+0000", 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015326200000000001 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845969+0000", 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7296000000000001e-05 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.004 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862827+0000", 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016857351999999999 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862890+0000", 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2997999999999996e-05 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=66, reqid=osd.12.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.102507218, 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017965090999999999, 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847246+0000", 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:35.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847298+0000", 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2677999999999999e-05 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847412+0000", 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113403 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847437+0000", 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4973e-05 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857022+0000", 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0095850889999999998 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857045+0000", 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3796000000000001e-05 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865168+0000", 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0081221260000000003 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865212+0000", 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4691999999999998e-05 2024-09-16T06:49:35.010 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=108, reqid=osd.14.0:108, at_version=72'100, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.099408206, 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015047932999999999, 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.011 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.012 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:35.012 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850339+0000", 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999928 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.013 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850413+0000", 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3508000000000002e-05 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850503+0000", 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0303999999999995e-05 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850526+0000", 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2595999999999998e-05 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850694+0000", 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168367 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850718+0000", 2024-09-16T06:49:35.015 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.44e-05 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865344+0000", 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014625658999999999 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865394+0000", 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0074999999999997e-05 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.016 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=109, reqid=osd.14.0:109, at_version=72'102, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 35.095760638000002, 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013551083, 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853992+0000", 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854050+0000", 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8495000000000002e-05 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854252+0000", 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020159999999999999 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854296+0000", 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4438999999999999e-05 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862592+0000", 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082959690000000003 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862662+0000", 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9210999999999999e-05 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867494+0000", 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048324029999999999 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867545+0000", 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0838999999999999e-05 2024-09-16T06:49:35.021 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:559 3.5s0 3:af79444c:::smithi11731613-29:1d [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.994774479999997, 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014695526, 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:35.022 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 559 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.023 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954979+0000", 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954984+0000", 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5310000000000003e-06 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954994+0000", 2024-09-16T06:49:35.024 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7249999999999992e-06 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.955000+0000", 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7569999999999998e-06 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967068+0000", 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012068096 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:35.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967239+0000", 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017012400000000001 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.969675+0000", 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024368660000000002 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=192, reqid=osd.6.0:192, at_version=73'33, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.100968395000002, 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017679872999999999, 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:35.027 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848785+0000", 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848820+0000", 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4931999999999997e-05 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849087+0000", 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026683499999999999 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849103+0000", 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6813e-05 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855706+0000", 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066022570000000003 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855730+0000", 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4661999999999999e-05 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866426+0000", 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010695893999999999 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866466+0000", 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9632999999999998e-05 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.030 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.098842492999999, 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015506908999999999, 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.031 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850911+0000", 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850952+0000", 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0722000000000001e-05 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851117+0000", 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000164599 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851132+0000", 2024-09-16T06:49:35.033 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5255e-05 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854555+0000", 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034234320000000001 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854579+0000", 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.034 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866327+0000", 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011748364000000001 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866419+0000", 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1624000000000002e-05 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.035 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 73/32 ECSubWrite(tid=107, reqid=osd.10.0:107, at_version=73'90, trim_to=0'0, roll_forward_to=72'88))", 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 34.095590022000003, 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012199229000000001, 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.036 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854217+0000", 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3168000000000003e-05 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.037 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854327+0000", 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109833 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854349+0000", 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2296000000000001e-05 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855253+0000", 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090369500000000004 2024-09-16T06:49:35.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855283+0000", 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9884999999999999e-05 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866242+0000", 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01095932 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866364+0000", 2024-09-16T06:49:35.039 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000121542 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:705 3.5s0 3:af79444c:::smithi11731613-29:head [append 0~524288] snapc 2b=[2b,2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 28.916018447999999, 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012998625, 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:35.040 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 705 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033733+0000", 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034535+0000", 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00080219500000000001 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034573+0000", 2024-09-16T06:49:35.042 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8473000000000002e-05 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034605+0000", 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1776999999999997e-05 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034755+0000", 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000149641 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.043 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.035008+0000", 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025373600000000001 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.039658+0000", 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0046500159999999999 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046264+0000", 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066053680000000004 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.044 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046596+0000", 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000331983 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046735+0000", 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013881599999999999 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 94/32 ECSubWrite(tid=165, reqid=osd.14.0:165, at_version=94'158, trim_to=0'0, roll_forward_to=92'156))", 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 13.014064661999999, 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012800693, 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:35.046 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935689+0000", 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935746+0000", 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7460999999999998e-05 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.047 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935852+0000", 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105815 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935875+0000", 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2886000000000001e-05 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936046+0000", 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000170929 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.048 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936070+0000", 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4746e-05 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.948119+0000", 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012048432 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.948490+0000", 2024-09-16T06:49:35.049 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037153400000000002 2024-09-16T06:49:35.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.050 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.050 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:35.050 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:35.050 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:35.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T06:49:35.052 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:35.050+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:35.153 INFO:tasks.ceph.osd.15.smithi117.stderr:2024-09-16T06:49:35.150+0000 7f3978c0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26547) UID: 0 2024-09-16T06:49:35.167 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:35.168 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:35.181 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T06:49:35.253 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:35.255+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:35.260 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:35.260 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:35.261 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:35.299 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:35.299 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:35.309 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:35.309 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:35.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T06:49:35.324 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:35.324 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:35.351 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:35.351 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:35.353 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:35.355+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:35.380 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:35.381 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:35.410 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:35.410 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:35.439 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:35.439 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:35.454 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:35.454 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:35.454 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:35.454+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:35.464 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:35.465 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:35.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T06:49:35.489 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:35.489 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:35.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:35.491+0000 7f2e9ddc6640 0 -- 172.21.15.100:0/578594856 >> [v2:172.21.15.115:6816/693497656,v1:172.21.15.115:6817/693497656] conn(0x7f2e982193d0 msgr2=0x7f2e9821b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-16T06:49:35.557 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-16T06:49:35.554+0000 7f8960d62640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26178) UID: 0 2024-09-16T06:49:35.604 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:35.604 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:35.618 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T06:49:35.658 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:35.658+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:35.671 INFO:tasks.rados.rados.0.smithi117.stdout:372: done (7 left) 2024-09-16T06:49:35.671 INFO:tasks.rados.rados.0.smithi117.stdout:393: done (6 left) 2024-09-16T06:49:35.671 INFO:tasks.rados.rados.0.smithi117.stdout:394: done (5 left) 2024-09-16T06:49:35.671 INFO:tasks.rados.rados.0.smithi117.stdout:395: done (4 left) 2024-09-16T06:49:35.671 INFO:tasks.rados.rados.0.smithi117.stdout:396: done (3 left) 2024-09-16T06:49:35.672 INFO:tasks.rados.rados.0.smithi117.stdout:397: done (2 left) 2024-09-16T06:49:35.672 INFO:tasks.rados.rados.0.smithi117.stdout:399: done (1 left) 2024-09-16T06:49:35.672 INFO:tasks.rados.rados.0.smithi117.stdout: waiting on 1 2024-09-16T06:49:35.720 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:35.720 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:35.740 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:35.740 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:35.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T06:49:35.759 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:35.759 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:35.759 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:35.758+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:35.762 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:35.762 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:49:35.860 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:35.857+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:35.879 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:35.879 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:35.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T06:49:35.961 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:35.958+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:36.056 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.056 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.062 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:36.058+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:36.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T06:49:36.163 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:36.158+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:36.191 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.191 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.205 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T06:49:36.263 INFO:tasks.ceph.osd.3.smithi117.stderr:2024-09-16T06:49:36.262+0000 7fa11604d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26543) UID: 0 2024-09-16T06:49:36.329 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.330 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-16T06:49:36.391 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:36.395+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:36.460 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.461 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.465 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T06:49:36.467+0000 7f5c1e75f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26295) UID: 0 2024-09-16T06:49:36.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-16T06:49:36.566 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-16T06:49:36.567+0000 7fd869589640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26293) UID: 0 2024-09-16T06:49:36.639 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.639 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.653 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-16T06:49:36.667 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:36.666+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:36.767 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-16T06:49:36.766+0000 7f106e406640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26180) UID: 0 2024-09-16T06:49:36.792 INFO:tasks.rados.rados.0.smithi117.stdout:398: finishing write tid 1 to smithi11731613-15 2024-09-16T06:49:36.792 INFO:tasks.rados.rados.0.smithi117.stdout:398: finishing write tid 2 to smithi11731613-15 2024-09-16T06:49:36.796 INFO:tasks.rados.rados.0.smithi117.stdout:398: finishing write tid 3 to smithi11731613-15 2024-09-16T06:49:36.796 INFO:tasks.rados.rados.0.smithi117.stdout:398: oid 15 updating version 0 to 204 2024-09-16T06:49:36.796 INFO:tasks.rados.rados.0.smithi117.stdout:398: oid 15 version 204 is already newer than 203 2024-09-16T06:49:36.796 INFO:tasks.rados.rados.0.smithi117.stdout:update_object_version oid 15 v 204 (ObjNum 156 snap 42 seq_num 156) dirty exists 2024-09-16T06:49:36.796 INFO:tasks.rados.rados.0.smithi117.stdout:398: left oid 15 (ObjNum 156 snap 42 seq_num 156) 2024-09-16T06:49:36.796 INFO:tasks.rados.rados.0.smithi117.stdout:398: done (0 left) 2024-09-16T06:49:36.814 INFO:tasks.rados.rados.0.smithi117.stderr:0 errors. 2024-09-16T06:49:36.814 INFO:tasks.rados.rados.0.smithi117.stderr: 2024-09-16T06:49:36.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:49:36.819 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.820 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T06:49:36.879 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:36.883+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:36.968 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:36.969 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:36.969 INFO:tasks.ceph.osd.13.smithi107.stderr:2024-09-16T06:49:36.969+0000 7f677a6e8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26173) UID: 0 2024-09-16T06:49:36.983 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T06:49:37.070 INFO:tasks.ceph.osd.12.smithi100.stderr:2024-09-16T06:49:37.071+0000 7fc397569640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26299) UID: 0 2024-09-16T06:49:37.130 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:37.130 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":109,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:49:36.834177+0000","last_up_change":"2024-09-16T06:49:10.837870+0000","last_in_change":"2024-09-16T06:49:04.862161+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.97786140441894531,"average_primary_affinity":0.98000001907348633,"average_primary_affinity_weighted":0.98000001907348633}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T06:48:08.489111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.97786140441894531,"average_primary_affinity":0.98000001907348633,"average_primary_affinity_weighted":0.98000001907348633}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-16T06:48:18.885917+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":72,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":0.7829742431640625,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":83,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6812","nonce":1081990569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6816","nonce":1081990569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6823","nonce":1081990569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1081990569},{"type":"v1","addr":"172.21.15.107:6820","nonce":1081990569}]},"public_addr":"172.21.15.107:6812/1081990569","cluster_addr":"172.21.15.107:6816/1081990569","heartbeat_back_addr":"172.21.15.107:6823/1081990569","heartbeat_front_addr":"172.21.15.107:6820/1081990569","state":["exists"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":0.645782470703125,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":0.5874481201171875,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":0.8076019287109375,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":0.7168731689453125,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.319627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:31.889358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.542092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:28.191679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:17.579785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:48:03.476690+0000","dead_epoch":83},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:21.476717+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:11.670056+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:19.668799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:25.819931+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:04.194869+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.647481+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.898585+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.022637+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.810301+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:33.012714+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":1,"to":2},{"from":15,"to":14}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":21,"length":1},{"begin":28,"length":2},{"begin":31,"length":3},{"begin":41,"length":4},{"begin":46,"length":1},{"begin":51,"length":1},{"begin":53,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:49:37.148 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:37.148 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:37.149 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:49:37.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T06:49:37.171 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:37.170+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:37.248 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 13] out_osds: [] dead_osds: [5] live_osds: [0, 4, 8, 12, 1, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-16T06:49:37.248 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T06:49:37.248 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-16T06:49:37.248 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-09-16T06:49:37.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:49:37.272 INFO:tasks.ceph.osd.9.smithi107.stderr:2024-09-16T06:49:37.269+0000 7fd099280640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26171) UID: 0 2024-09-16T06:49:37.279 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:37.279 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:37.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T06:49:37.372 INFO:tasks.ceph.osd.8.smithi100.stderr:2024-09-16T06:49:37.375+0000 7f9a01133640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26297) UID: 0 2024-09-16T06:49:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:37.460 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:37.473 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:37.470+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:37.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T06:49:37.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:49:37.530 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-16T06:49:36.834102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147030579,"num_objects":131,"num_object_clones":80,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":9,"num_read":577,"num_read_kb":283718,"num_write":1007,"num_write_kb":157846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":160563200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1884,"ondisk_log_size":1884,"up":129,"acting":129,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":803472,"kb_used_data":273540,"kb_used_omap":24,"kb_used_meta":529639,"kb_avail":1509145968,"statfs":{"total":1546188226560,"available":1545365471232,"internally_reserved":0,"allocated":280104960,"data_stored":419174809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":542350432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15433728,"num_objects":14,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":26,"num_read_kb":37345,"num_write":55,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":34668544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.009890"},"pg_stats":[{"pgid":"3.f","version":"101'162","reported_seq":437,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-16T06:49:31.018098+0000","last_change":"2024-09-16T06:49:31.018098+0000","last_active":"2024-09-16T06:49:30.006133+0000","last_peered":"2024-09-16T06:49:29.546894+0000","last_clean":"2024-09-16T06:49:29.546894+0000","last_became_active":"2024-09-16T06:49:05.959106+0000","last_became_peered":"2024-09-16T06:49:05.959106+0000","last_unstale":"2024-09-16T06:49:31.018098+0000","last_undegraded":"2024-09-16T06:49:31.018098+0000","last_fullsized":"2024-09-16T06:49:31.018098+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'154","last_scrub_stamp":"2024-09-16T06:49:24.451768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.451768+0000","objects_scrubbed":7,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:43.815768+0000","scrub_duration":0.010205087,"objects_trimmed":0,"snaptrim_duration":0.016389463,"stat_sum":{"num_bytes":10158080,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":4479,"num_write":48,"num_write_kb":11488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":7307264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,2,3,0],"acting":[11,12,4,14,2,2147483647,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"86'32","reported_seq":245,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:34.837702+0000","last_change":"2024-09-16T06:49:12.265575+0000","last_active":"2024-09-16T06:49:34.837702+0000","last_peered":"2024-09-16T06:49:34.837702+0000","last_clean":"2024-09-16T06:49:09.884100+0000","last_became_active":"2024-09-16T06:49:12.265575+0000","last_became_peered":"2024-09-16T06:49:12.265575+0000","last_unstale":"2024-09-16T06:49:34.837702+0000","last_undegraded":"2024-09-16T06:49:11.860617+0000","last_fullsized":"2024-09-16T06:49:11.860421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018081014999999999,"stat_sum":{"num_bytes":3735552,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":5284,"num_write":30,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":6127616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,2147483647],"acting":[8,14,6,11,13,0,2147483647],"avail_no_missing":["8(0)","0(5)","6(2)","11(3)","13(4)","14(1)"],"object_location_counts":[{"shards":"0(5),6(2),8(0),11(3),13(4),14(1)","objects":4}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.d","version":"106'102","reported_seq":337,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:33.829198+0000","last_change":"2024-09-16T06:49:11.864865+0000","last_active":"2024-09-16T06:49:33.829198+0000","last_peered":"2024-09-16T06:49:33.829198+0000","last_clean":"2024-09-16T06:49:09.863855+0000","last_became_active":"2024-09-16T06:49:11.864865+0000","last_became_peered":"2024-09-16T06:49:11.864865+0000","last_unstale":"2024-09-16T06:49:33.829198+0000","last_undegraded":"2024-09-16T06:49:11.858666+0000","last_fullsized":"2024-09-16T06:49:11.858254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019226413000000001,"stat_sum":{"num_bytes":11059200,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":23,"num_read_kb":5675,"num_write":46,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7372800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,2147483647],"acting":[7,3,0,12,9,13,2147483647],"avail_no_missing":["3(1)","0(2)","7(0)","9(4)","12(3)","13(5)"],"object_location_counts":[{"shards":"0(2),3(1),7(0),9(4),12(3),13(5)","objects":9}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"90'122","reported_seq":370,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:31.016924+0000","last_change":"2024-09-16T06:49:11.863339+0000","last_active":"2024-09-16T06:49:31.016924+0000","last_peered":"2024-09-16T06:49:31.016924+0000","last_clean":"2024-09-16T06:49:09.898987+0000","last_became_active":"2024-09-16T06:49:11.863339+0000","last_became_peered":"2024-09-16T06:49:11.863339+0000","last_unstale":"2024-09-16T06:49:31.016924+0000","last_undegraded":"2024-09-16T06:49:11.858719+0000","last_fullsized":"2024-09-16T06:49:11.858364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.034010460999999999,"stat_sum":{"num_bytes":16154624,"num_objects":8,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":37,"num_read_kb":73697,"num_write":49,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":25968640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,2147483647,12,4,1,9],"acting":[11,13,2147483647,12,4,1,9],"avail_no_missing":["11(0)","1(5)","4(4)","9(6)","12(3)","13(1)"],"object_location_counts":[{"shards":"1(5),4(4),9(6),11(0),12(3),13(1)","objects":8}],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"107'40","reported_seq":157,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.858930+0000","last_change":"2024-09-16T06:49:32.835071+0000","last_active":"2024-09-16T06:49:34.858930+0000","last_peered":"2024-09-16T06:49:34.858930+0000","last_clean":"2024-09-16T06:49:34.858930+0000","last_became_active":"2024-09-16T06:49:05.925903+0000","last_became_peered":"2024-09-16T06:49:05.925903+0000","last_unstale":"2024-09-16T06:49:34.858930+0000","last_undegraded":"2024-09-16T06:49:34.858930+0000","last_fullsized":"2024-09-16T06:49:34.858930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'31","last_scrub_stamp":"2024-09-16T06:49:25.350164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:25.350164+0000","objects_scrubbed":3,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:43.192516+0000","scrub_duration":0.0082238669999999993,"objects_trimmed":0,"snaptrim_duration":0.00123594,"stat_sum":{"num_bytes":409600,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":1515,"num_write":28,"num_write_kb":496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,10,3,8,7,14],"acting":[13,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.a","version":"105'244","reported_seq":505,"reported_epoch":106,"state":"peering","last_fresh":"2024-09-16T06:49:33.830702+0000","last_change":"2024-09-16T06:49:33.831281+0000","last_active":"2024-09-16T06:49:33.830702+0000","last_peered":"2024-09-16T06:49:33.034204+0000","last_clean":"2024-09-16T06:49:30.014813+0000","last_became_active":"2024-09-16T06:49:32.858571+0000","last_became_peered":"2024-09-16T06:49:32.858571+0000","last_unstale":"2024-09-16T06:49:33.830702+0000","last_undegraded":"2024-09-16T06:49:33.830702+0000","last_fullsized":"2024-09-16T06:49:33.830702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"98'224","last_scrub_stamp":"2024-09-16T06:49:26.790816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.790816+0000","objects_scrubbed":14,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:46.325635+0000","scrub_duration":0.44054632900000001,"objects_trimmed":0,"snaptrim_duration":0.040880347999999997,"stat_sum":{"num_bytes":24035328,"num_objects":16,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":41,"num_read_kb":13839,"num_write":60,"num_write_kb":24336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":31260672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,9,8,15,11],"acting":[3,7,6,9,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7,8,9,10,11,15],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.9","version":"105'60","reported_seq":302,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.835957+0000","last_change":"2024-09-16T06:49:32.844251+0000","last_active":"2024-09-16T06:49:34.835957+0000","last_peered":"2024-09-16T06:49:34.835957+0000","last_clean":"2024-09-16T06:49:34.835957+0000","last_became_active":"2024-09-16T06:49:31.031566+0000","last_became_peered":"2024-09-16T06:49:31.031566+0000","last_unstale":"2024-09-16T06:49:34.835957+0000","last_undegraded":"2024-09-16T06:49:34.835957+0000","last_fullsized":"2024-09-16T06:49:34.835957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:48.756986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013473433,"stat_sum":{"num_bytes":8454144,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":18150,"num_write":22,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":17514496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,13,4,15,9,1],"acting":[12,2,13,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.8","version":"105'45","reported_seq":238,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:34.834040+0000","last_change":"2024-09-16T06:49:25.247568+0000","last_active":"2024-09-16T06:49:34.834040+0000","last_peered":"2024-09-16T06:49:34.834040+0000","last_clean":"2024-09-16T06:49:09.868266+0000","last_became_active":"2024-09-16T06:49:25.217434+0000","last_became_peered":"2024-09-16T06:49:25.217434+0000","last_unstale":"2024-09-16T06:49:34.834040+0000","last_undegraded":"2024-09-16T06:49:24.957956+0000","last_fullsized":"2024-09-16T06:49:24.957445+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:45.535736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015152634999999999,"stat_sum":{"num_bytes":5472256,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":8,"num_read_kb":4,"num_write":55,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4472832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,2147483647,14,2],"acting":[6,1,0,4,2147483647,14,2],"avail_no_missing":["6(0)","0(2)","1(1)","2(6)","4(3)","14(5)","15(5)"],"object_location_counts":[{"shards":"0(2),1(1),2(6),4(3),6(0),14(5)","objects":7}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"28'157","reported_seq":300,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.834279+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:49:34.834279+0000","last_peered":"2024-09-16T06:49:34.834279+0000","last_clean":"2024-09-16T06:49:34.834279+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:49:34.834279+0000","last_undegraded":"2024-09-16T06:49:34.834279+0000","last_fullsized":"2024-09-16T06:49:34.834279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":90,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.017064+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:49:31.017064+0000","last_peered":"2024-09-16T06:49:31.017064+0000","last_clean":"2024-09-16T06:49:31.017064+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:49:31.017064+0000","last_undegraded":"2024-09-16T06:49:31.017064+0000","last_fullsized":"2024-09-16T06:49:31.017064+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"101'195","reported_seq":382,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-16T06:49:31.830843+0000","last_change":"2024-09-16T06:49:31.831278+0000","last_active":"2024-09-16T06:49:30.009235+0000","last_peered":"2024-09-16T06:49:29.640414+0000","last_clean":"2024-09-16T06:49:29.640414+0000","last_became_active":"2024-09-16T06:48:20.888283+0000","last_became_peered":"2024-09-16T06:48:20.888283+0000","last_unstale":"2024-09-16T06:49:31.830843+0000","last_undegraded":"2024-09-16T06:49:31.830843+0000","last_fullsized":"2024-09-16T06:49:31.830843+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:36.741864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.02739635,"stat_sum":{"num_bytes":12500992,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":29,"num_read_kb":6281,"num_write":133,"num_write_kb":12032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,7,0],"acting":[14,12,3,4,15,2147483647,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,4,7,10,11,12,15],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":103,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T06:49:32.832704+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:49:32.832704+0000","last_peered":"2024-09-16T06:49:32.832704+0000","last_clean":"2024-09-16T06:49:32.832704+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:49:32.832704+0000","last_undegraded":"2024-09-16T06:49:32.832704+0000","last_fullsized":"2024-09-16T06:49:32.832704+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"107'92","reported_seq":178,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.856764+0000","last_change":"2024-09-16T06:49:32.861640+0000","last_active":"2024-09-16T06:49:34.856764+0000","last_peered":"2024-09-16T06:49:34.856764+0000","last_clean":"2024-09-16T06:49:34.856764+0000","last_became_active":"2024-09-16T06:49:05.936871+0000","last_became_peered":"2024-09-16T06:49:05.936871+0000","last_unstale":"2024-09-16T06:49:34.856764+0000","last_undegraded":"2024-09-16T06:49:34.856764+0000","last_fullsized":"2024-09-16T06:49:34.856764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'75","last_scrub_stamp":"2024-09-16T06:49:24.392872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.392872+0000","objects_scrubbed":5,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:49.469666+0000","scrub_duration":0.0091732289999999998,"objects_trimmed":0,"snaptrim_duration":0.027918088000000001,"stat_sum":{"num_bytes":5865472,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":17,"num_read_kb":16711,"num_write":30,"num_write_kb":6320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5292032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.017107+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:49:31.017107+0000","last_peered":"2024-09-16T06:49:31.017107+0000","last_clean":"2024-09-16T06:49:31.017107+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:49:31.017107+0000","last_undegraded":"2024-09-16T06:49:31.017107+0000","last_fullsized":"2024-09-16T06:49:31.017107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"107'67","reported_seq":278,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.876206+0000","last_change":"2024-09-16T06:49:32.831990+0000","last_active":"2024-09-16T06:49:34.876206+0000","last_peered":"2024-09-16T06:49:34.876206+0000","last_clean":"2024-09-16T06:49:34.876206+0000","last_became_active":"2024-09-16T06:48:21.124782+0000","last_became_peered":"2024-09-16T06:48:21.124782+0000","last_unstale":"2024-09-16T06:49:34.876206+0000","last_undegraded":"2024-09-16T06:49:34.876206+0000","last_fullsized":"2024-09-16T06:49:34.876206+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.918920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013075929999999999,"stat_sum":{"num_bytes":5079040,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":775,"num_write":23,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":94,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T06:49:32.840680+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:49:32.840680+0000","last_peered":"2024-09-16T06:49:32.840680+0000","last_clean":"2024-09-16T06:49:32.840680+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:49:32.840680+0000","last_undegraded":"2024-09-16T06:49:32.840680+0000","last_fullsized":"2024-09-16T06:49:32.840680+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"104'158","reported_seq":376,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.880654+0000","last_change":"2024-09-16T06:49:31.859053+0000","last_active":"2024-09-16T06:49:31.880654+0000","last_peered":"2024-09-16T06:49:31.880654+0000","last_clean":"2024-09-16T06:49:31.880654+0000","last_became_active":"2024-09-16T06:48:21.121223+0000","last_became_peered":"2024-09-16T06:48:21.121223+0000","last_unstale":"2024-09-16T06:49:31.880654+0000","last_undegraded":"2024-09-16T06:49:31.880654+0000","last_fullsized":"2024-09-16T06:49:31.880654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.389211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022564463,"stat_sum":{"num_bytes":13238272,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":62,"num_read_kb":25509,"num_write":88,"num_write_kb":13760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.5","version":"82'12","reported_seq":242,"reported_epoch":104,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:31.884037+0000","last_change":"2024-09-16T06:49:31.885499+0000","last_active":"2024-09-16T06:49:31.884037+0000","last_peered":"2024-09-16T06:49:31.884037+0000","last_clean":"2024-09-16T06:49:09.867185+0000","last_became_active":"2024-09-16T06:49:31.848579+0000","last_became_peered":"2024-09-16T06:49:31.848579+0000","last_unstale":"2024-09-16T06:49:31.884037+0000","last_undegraded":"2024-09-16T06:49:31.837321+0000","last_fullsized":"2024-09-16T06:49:31.835689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017739370000000001,"stat_sum":{"num_bytes":2555904,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":17765,"num_write":7,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,2147483647,2,7,9,8],"acting":[15,13,2147483647,2,7,9,8],"avail_no_missing":["15(0)","1(5)","2(3)","7(4)","8(6)","9(5)","13(1)"],"object_location_counts":[{"shards":"2(3),7(4),8(6),9(5),13(1),15(0)","objects":2}],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.024656+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:49:31.024656+0000","last_peered":"2024-09-16T06:49:31.024656+0000","last_clean":"2024-09-16T06:49:31.024656+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:49:31.024656+0000","last_undegraded":"2024-09-16T06:49:31.024656+0000","last_fullsized":"2024-09-16T06:49:31.024656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"104'185","reported_seq":446,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.877309+0000","last_change":"2024-09-16T06:49:31.853957+0000","last_active":"2024-09-16T06:49:31.877309+0000","last_peered":"2024-09-16T06:49:31.877309+0000","last_clean":"2024-09-16T06:49:31.877309+0000","last_became_active":"2024-09-16T06:49:05.920150+0000","last_became_peered":"2024-09-16T06:49:05.920150+0000","last_unstale":"2024-09-16T06:49:31.877309+0000","last_undegraded":"2024-09-16T06:49:31.877309+0000","last_fullsized":"2024-09-16T06:49:31.877309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:38.017254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022284939,"stat_sum":{"num_bytes":7028736,"num_objects":10,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":41,"num_read_kb":30973,"num_write":74,"num_write_kb":8560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9486336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,2,6,10,13],"acting":[1,12,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+undersized","last_fresh":"2024-09-16T06:49:34.837869+0000","last_change":"2024-09-16T06:49:11.861382+0000","last_active":"2024-09-16T06:49:34.837869+0000","last_peered":"2024-09-16T06:49:34.837869+0000","last_clean":"2024-09-16T06:49:09.866035+0000","last_became_active":"2024-09-16T06:49:11.861382+0000","last_became_peered":"2024-09-16T06:49:11.861382+0000","last_unstale":"2024-09-16T06:49:34.837869+0000","last_undegraded":"2024-09-16T06:49:34.837869+0000","last_fullsized":"2024-09-16T06:49:11.860463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8],"acting":[8],"avail_no_missing":["8"],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"106'169","reported_seq":207,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-16T06:49:33.902387+0000","last_change":"2024-09-16T06:49:32.872905+0000","last_active":"2024-09-16T06:49:33.902387+0000","last_peered":"2024-09-16T06:49:33.902387+0000","last_clean":"2024-09-16T06:49:33.902387+0000","last_became_active":"2024-09-16T06:49:05.952263+0000","last_became_peered":"2024-09-16T06:49:05.952263+0000","last_unstale":"2024-09-16T06:49:33.902387+0000","last_undegraded":"2024-09-16T06:49:33.902387+0000","last_fullsized":"2024-09-16T06:49:33.902387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038875240999999998,"stat_sum":{"num_bytes":17760256,"num_objects":13,"num_object_clones":9,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":44,"num_read_kb":47767,"num_write":100,"num_write_kb":19216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":34914304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":89,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.025403+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:49:31.025403+0000","last_peered":"2024-09-16T06:49:31.025403+0000","last_clean":"2024-09-16T06:49:31.025403+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:49:31.025403+0000","last_undegraded":"2024-09-16T06:49:31.025403+0000","last_fullsized":"2024-09-16T06:49:31.025403+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"105'39","reported_seq":44,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-16T06:49:33.829589+0000","last_change":"2024-09-16T06:49:32.844621+0000","last_active":"2024-09-16T06:49:33.829589+0000","last_peered":"2024-09-16T06:49:33.829589+0000","last_clean":"2024-09-16T06:49:33.829589+0000","last_became_active":"2024-09-16T06:49:31.036738+0000","last_became_peered":"2024-09-16T06:49:31.036738+0000","last_unstale":"2024-09-16T06:49:33.829589+0000","last_undegraded":"2024-09-16T06:49:33.829589+0000","last_fullsized":"2024-09-16T06:49:33.829589+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011710752,"stat_sum":{"num_bytes":2932736,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":15176,"num_write":9,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":6307840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.017805+0000","last_change":"2024-09-16T06:49:12.584897+0000","last_active":"2024-09-16T06:49:31.017805+0000","last_peered":"2024-09-16T06:49:31.017805+0000","last_clean":"2024-09-16T06:49:31.017805+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:49:31.017805+0000","last_undegraded":"2024-09-16T06:49:31.017805+0000","last_fullsized":"2024-09-16T06:49:31.017805+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:12.584669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:12.584669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:13.936208+0000","scrub_duration":0.0037478810000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":146440192,"num_objects":127,"num_object_clones":80,"num_object_copies":889,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":9,"num_read":441,"num_read_kb":283600,"num_write":802,"num_write_kb":154384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":160563200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258154496,"data_stored":408563712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1724,"ondisk_log_size":1724,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":15,"acting":15,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247827,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46724,"kb_used_data":15604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325116,"statfs":{"total":96636764160,"available":96588918784,"internally_reserved":0,"allocated":15978496,"data_stored":23167488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":45.249000000000002,"5min":45.249000000000002,"15min":45.249000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":780.85000000000002,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":9.7080000000000002,"5min":9.7080000000000002,"15min":9.7080000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":164.398,"5min":164.398,"15min":164.398},"last":1.0349999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.113}]},{"osd":2,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.238},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.21}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":13.01,"5min":13.01,"15min":13.01},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":225.697,"5min":225.697,"15min":225.697},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.0900000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.73699999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":348.30599999999998},{"interface":"front","average":{"1min":12.073,"5min":12.073,"15min":12.073},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":197.559,"5min":197.559,"15min":197.559},"last":1.3979999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.2689999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1919999999999999}]}]},{"osd":14,"up_from":23,"seq":98784247829,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47320,"kb_used_data":16196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324520,"statfs":{"total":96636764160,"available":96588308480,"internally_reserved":0,"allocated":16584704,"data_stored":24910762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":27.170999999999999,"5min":27.170999999999999,"15min":27.170999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":525.66800000000001,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":105.913}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.242}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.272}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.95399999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.333},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.226}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.151}]},{"osd":12,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.464}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.552},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.3640000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":360.42200000000003,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.413},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":3.7709999999999999,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.526}]}]},{"osd":11,"up_from":23,"seq":98784247828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70676,"kb_used_data":21504,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94301164,"statfs":{"total":96636764160,"available":96564391936,"internally_reserved":0,"allocated":22020096,"data_stored":36871082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":50.844000000000001,"5min":50.844000000000001,"15min":50.844000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":949.71199999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":48.140999999999998,"5min":48.140999999999998,"15min":48.140999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":894.49900000000002,"5min":894.49900000000002,"15min":894.49900000000002},"last":0.84999999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.163},{"interface":"front","average":{"1min":4.8739999999999997,"5min":4.8739999999999997,"15min":4.8739999999999997},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":71.503,"5min":71.503,"15min":71.503},"last":0.95899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.879},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.7400000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.139}]},{"osd":10,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6469999999999998,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.006},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.1100000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.044},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.1830000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.274},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.298}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313237,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48408,"kb_used_data":17288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323432,"statfs":{"total":96636764160,"available":96587194368,"internally_reserved":0,"allocated":17702912,"data_stored":24518377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3660000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.298},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1819999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":2.052},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":2.0699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.113},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.86399999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.73399999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.0620000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":42.863,"5min":42.863,"15min":42.863},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":502.57400000000001,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":41.012,"5min":41.012,"15min":41.012},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":502.62200000000001,"5min":502.62200000000001,"15min":502.62200000000001},"last":0.95199999999999996}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.393}]},{"osd":12,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.347},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.234}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.2869999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313237,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":29508,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":30216192,"data_stored":39507882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":8.6449999999999996,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.96799999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":28.696999999999999,"5min":28.696999999999999,"15min":28.696999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":385.83800000000002,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.167}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.90800000000000003}]},{"osd":8,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.68300000000000005},{"interface":"front","average":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":96.432000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.0800000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.2330000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":10.728,"5min":10.728,"15min":10.728},"last":1.254},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":3.8069999999999999,"5min":3.8069999999999999,"15min":3.8069999999999999},"last":1.1859999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.3180000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2,"5min":2,"15min":2},"last":1.379}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39748,"kb_used_data":8632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332092,"statfs":{"total":96636764160,"available":96596062208,"internally_reserved":0,"allocated":8839168,"data_stored":11416608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":39.485999999999997,"5min":39.485999999999997,"15min":39.485999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":734.39200000000005,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.113},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.96599999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.0149999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.754},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":32.578000000000003,"5min":32.578000000000003,"15min":32.578000000000003},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":596.70600000000002,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.98399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.079},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":0.89100000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.7149999999999999,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":1.0620000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.169},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.036}]}]},{"osd":4,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49304,"kb_used_data":18188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94322536,"statfs":{"total":96636764160,"available":96586276864,"internally_reserved":0,"allocated":18624512,"data_stored":30310126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.96799999999999997},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":14.775,"5min":14.775,"15min":14.775},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":265.48099999999999,"5min":265.48099999999999,"15min":265.48099999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.022}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.133},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":1.1080000000000001}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":31.298999999999999,"5min":31.298999999999999,"15min":31.298999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":576.29200000000003,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.038}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.755},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.83399999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.99,"15min":1.99},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":20.283999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":16.338999999999999,"5min":16.338999999999999,"15min":16.338999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":306.85599999999999,"5min":306.85599999999999,"15min":306.85599999999999},"last":0.94199999999999995}]},{"osd":10,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.2390000000000001},{"interface":"front","average":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":21.588999999999999,"5min":21.588999999999999,"15min":21.588999999999999},"last":1.196}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.337}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.226},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.119}]},{"osd":15,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.306},{"interface":"front","average":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":20.437999999999999,"5min":20.437999999999999,"15min":20.437999999999999},"last":1.246}]}]},{"osd":5,"up_from":22,"seq":94489280527,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34536,"kb_used_data":7508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337304,"statfs":{"total":96636764160,"available":96601399296,"internally_reserved":0,"allocated":7688192,"data_stored":10847401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,8,10,11,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1579999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]}]},{"osd":6,"up_from":22,"seq":94489280533,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49308,"kb_used_data":18196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94322532,"statfs":{"total":96636764160,"available":96586272768,"internally_reserved":0,"allocated":18632704,"data_stored":28162506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.6000000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.74}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.0109999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.4890000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":13.279,"5min":13.279,"15min":13.279},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":231.90700000000001,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.6859999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":2.0470000000000002},{"interface":"front","average":{"1min":18.225000000000001,"5min":18.225000000000001,"15min":18.225000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":323.55500000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.4510000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.84999999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.073},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.0920000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":2.1829999999999998},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.5760000000000001}]}]},{"osd":7,"up_from":22,"seq":94489280535,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49560,"kb_used_data":18436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94322280,"statfs":{"total":96636764160,"available":96586014720,"internally_reserved":0,"allocated":18878464,"data_stored":24911598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.59699999999999998},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.98199999999999998},{"interface":"front","average":{"1min":29.103999999999999,"5min":29.103999999999999,"15min":29.103999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":527.10299999999995,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.878}]},{"osd":2,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":6.1749999999999998,"5min":6.1749999999999998,"15min":6.1749999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":136.72399999999999,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.91100000000000003}]},{"osd":3,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":20.370999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":508.52100000000002,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.012}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":30.073,"5min":30.073,"15min":30.073},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":415.30099999999999,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":37.726999999999997,"5min":37.726999999999997,"15min":37.726999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":502.55200000000002,"5min":502.55200000000002,"15min":502.55200000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":20.564,"5min":20.564,"15min":20.564},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":502.39600000000002,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.0329999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":12,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.05},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.1220000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.103},{"interface":"front","average":{"1min":21.131,"5min":21.131,"15min":21.131},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":514.07799999999997,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.2010000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44532,"kb_used_data":13408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327308,"statfs":{"total":96636764160,"available":96591163392,"internally_reserved":0,"allocated":13729792,"data_stored":22023918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.008},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.38900000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":10.706,"5min":10.706,"15min":10.706},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":177.92400000000001,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.754},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.70599999999999996}]},{"osd":6,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":27.687000000000001,"5min":27.687000000000001,"15min":27.687000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":449.88,"5min":449.88,"15min":449.88},"last":0.499},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.1359999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.59799999999999998}]},{"osd":9,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.903,"5min":1.903,"15min":1.903},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":12.366,"5min":12.366,"15min":12.366},"last":1.3080000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:14 2024","interfaces":[{"interface":"back","average":{"1min":56.750999999999998,"5min":56.750999999999998,"15min":56.750999999999998},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":998.42399999999998,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.331}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":0.77000000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.5869999999999997,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3200000000000001}]}]},{"osd":12,"up_from":22,"seq":94489280534,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":20424,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":20914176,"data_stored":33300206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":0.64900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90000000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.4119999999999999,"5min":3.4119999999999999,"15min":3.4119999999999999},"last":151.34800000000001},{"interface":"front","average":{"1min":38.109000000000002,"5min":38.109000000000002,"15min":38.109000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":708.49699999999996,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.627}]},{"osd":4,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":33.972000000000001,"5min":33.972000000000001,"15min":33.972000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":630.75900000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.72099999999999997},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.86599999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.3330000000000002,"5min":4.3330000000000002,"15min":4.3330000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.95399999999999996}]},{"osd":11,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":50.134999999999998,"5min":50.134999999999998,"15min":50.134999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":980.38300000000004,"5min":980.38300000000004,"15min":980.38300000000004},"last":150.93299999999999}]},{"osd":13,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":229.631}]},{"osd":14,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.997},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.91900000000000004}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.93100000000000005}]}]},{"osd":1,"up_from":23,"seq":98784247830,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53044,"kb_used_data":21920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94318796,"statfs":{"total":96636764160,"available":96582447104,"internally_reserved":0,"allocated":22446080,"data_stored":37641130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":12.209,"5min":12.209,"15min":12.209},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":214.93299999999999,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.361},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":12.398,"5min":12.398,"15min":12.398},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":216.958,"5min":216.958,"15min":216.958},"last":1.036},{"interface":"front","average":{"1min":35.026000000000003,"5min":35.026000000000003,"15min":35.026000000000003},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":645.88099999999997,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":28.591000000000001,"5min":28.591000000000001,"15min":28.591000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":493.76100000000002,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.871},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3100000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.2809999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":37.526000000000003,"5min":37.526000000000003,"15min":37.526000000000003},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":687.10799999999995,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.258},{"interface":"front","average":{"1min":26.273,"5min":26.273,"15min":26.273},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":475.75700000000001,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.103},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.1970000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.3919999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.45},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.3380000000000001}]}]},{"osd":2,"up_from":23,"seq":98784247828,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46768,"kb_used_data":15648,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325072,"statfs":{"total":96636764160,"available":96588873728,"internally_reserved":0,"allocated":16023552,"data_stored":22241792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.879},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.72499999999999998}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.82399999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":19.053000000000001,"5min":19.053000000000001,"15min":19.053000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":361.36799999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.38200000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":35.137999999999998,"5min":35.137999999999998,"15min":35.137999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":646.98800000000006,"5min":646.98800000000006,"15min":646.98800000000006},"last":0.64100000000000001},{"interface":"front","average":{"1min":28.411999999999999,"5min":28.411999999999999,"15min":28.411999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":516.49000000000001,"5min":516.49000000000001,"15min":516.49000000000001},"last":0.89900000000000002}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.8239999999999998,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.66200000000000003}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":0.86499999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":9,"up_from":23,"seq":98784247827,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44212,"kb_used_data":13092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327628,"statfs":{"total":96636764160,"available":96591491072,"internally_reserved":0,"allocated":13406208,"data_stored":20861459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":50.427999999999997,"5min":50.427999999999997,"15min":50.427999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":941.91899999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.71399999999999997}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":104.761,"5min":104.761,"15min":104.761},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":20.274000000000001,"5min":20.274000000000001,"15min":20.274000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":363.67399999999998,"5min":363.67399999999998,"15min":363.67399999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.98199999999999998}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.0049999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":1.018}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.0549999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.042}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":49.017000000000003,"5min":49.017000000000003,"15min":49.017000000000003},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":750.83299999999997,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.077}]}]},{"osd":10,"up_from":23,"seq":98784247828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67160,"kb_used_data":17988,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94304680,"statfs":{"total":96636764160,"available":96567992320,"internally_reserved":0,"allocated":18419712,"data_stored":28482474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[1,2,3,4,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.93200000000000005},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":599.428},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.232}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.64900000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.071},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.1930000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.153}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.101}]},{"osd":9,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.198},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.99399999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6787072,"data_stored":10506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20905984,"data_stored":36741120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14659584,"data_stored":21512192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":25702400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17018880,"data_stored":29368320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7049216,"data_stored":10805248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16510976,"data_stored":26701824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":17272832,"data_stored":23969792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12124160,"data_stored":21082112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":12652544,"data_stored":21250048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16879616,"data_stored":27582464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480000,"data_stored":35971072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":19308544,"data_stored":32358400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28626944,"data_stored":38576128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":15044608,"data_stored":24010752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":14602240,"data_stored":22425600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:49:37.564 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:49:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:49:37.574 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:37.574+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:37.582 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-16T06:49:36.834102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147030579,"num_objects":131,"num_object_clones":80,"num_object_copies":897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":9,"num_read":577,"num_read_kb":283718,"num_write":1007,"num_write_kb":157846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":160563200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1884,"ondisk_log_size":1884,"up":129,"acting":129,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":803472,"kb_used_data":273540,"kb_used_omap":24,"kb_used_meta":529639,"kb_avail":1509145968,"statfs":{"total":1546188226560,"available":1545365471232,"internally_reserved":0,"allocated":280104960,"data_stored":419174809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25504,"internal_metadata":542350432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15433728,"num_objects":14,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":26,"num_read_kb":37345,"num_write":55,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":34668544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.009890"},"pg_stats":[{"pgid":"3.f","version":"101'162","reported_seq":437,"reported_epoch":103,"state":"remapped+peering","last_fresh":"2024-09-16T06:49:31.018098+0000","last_change":"2024-09-16T06:49:31.018098+0000","last_active":"2024-09-16T06:49:30.006133+0000","last_peered":"2024-09-16T06:49:29.546894+0000","last_clean":"2024-09-16T06:49:29.546894+0000","last_became_active":"2024-09-16T06:49:05.959106+0000","last_became_peered":"2024-09-16T06:49:05.959106+0000","last_unstale":"2024-09-16T06:49:31.018098+0000","last_undegraded":"2024-09-16T06:49:31.018098+0000","last_fullsized":"2024-09-16T06:49:31.018098+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'154","last_scrub_stamp":"2024-09-16T06:49:24.451768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.451768+0000","objects_scrubbed":7,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:43.815768+0000","scrub_duration":0.010205087,"objects_trimmed":0,"snaptrim_duration":0.016389463,"stat_sum":{"num_bytes":10158080,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":4479,"num_write":48,"num_write_kb":11488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":7307264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,12,4,14,2,3,0],"acting":[11,12,4,14,2,2147483647,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.e","version":"86'32","reported_seq":245,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:34.837702+0000","last_change":"2024-09-16T06:49:12.265575+0000","last_active":"2024-09-16T06:49:34.837702+0000","last_peered":"2024-09-16T06:49:34.837702+0000","last_clean":"2024-09-16T06:49:09.884100+0000","last_became_active":"2024-09-16T06:49:12.265575+0000","last_became_peered":"2024-09-16T06:49:12.265575+0000","last_unstale":"2024-09-16T06:49:34.837702+0000","last_undegraded":"2024-09-16T06:49:11.860617+0000","last_fullsized":"2024-09-16T06:49:11.860421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018081014999999999,"stat_sum":{"num_bytes":3735552,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":5284,"num_write":30,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":6127616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,14,6,11,13,0,2147483647],"acting":[8,14,6,11,13,0,2147483647],"avail_no_missing":["8(0)","0(5)","6(2)","11(3)","13(4)","14(1)"],"object_location_counts":[{"shards":"0(5),6(2),8(0),11(3),13(4),14(1)","objects":4}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.d","version":"106'102","reported_seq":337,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:33.829198+0000","last_change":"2024-09-16T06:49:11.864865+0000","last_active":"2024-09-16T06:49:33.829198+0000","last_peered":"2024-09-16T06:49:33.829198+0000","last_clean":"2024-09-16T06:49:09.863855+0000","last_became_active":"2024-09-16T06:49:11.864865+0000","last_became_peered":"2024-09-16T06:49:11.864865+0000","last_unstale":"2024-09-16T06:49:33.829198+0000","last_undegraded":"2024-09-16T06:49:11.858666+0000","last_fullsized":"2024-09-16T06:49:11.858254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019226413000000001,"stat_sum":{"num_bytes":11059200,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":23,"num_read_kb":5675,"num_write":46,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7372800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,12,9,13,2147483647],"acting":[7,3,0,12,9,13,2147483647],"avail_no_missing":["3(1)","0(2)","7(0)","9(4)","12(3)","13(5)"],"object_location_counts":[{"shards":"0(2),3(1),7(0),9(4),12(3),13(5)","objects":9}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"90'122","reported_seq":370,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:31.016924+0000","last_change":"2024-09-16T06:49:11.863339+0000","last_active":"2024-09-16T06:49:31.016924+0000","last_peered":"2024-09-16T06:49:31.016924+0000","last_clean":"2024-09-16T06:49:09.898987+0000","last_became_active":"2024-09-16T06:49:11.863339+0000","last_became_peered":"2024-09-16T06:49:11.863339+0000","last_unstale":"2024-09-16T06:49:31.016924+0000","last_undegraded":"2024-09-16T06:49:11.858719+0000","last_fullsized":"2024-09-16T06:49:11.858364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.034010460999999999,"stat_sum":{"num_bytes":16154624,"num_objects":8,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":37,"num_read_kb":73697,"num_write":49,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":25968640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,13,2147483647,12,4,1,9],"acting":[11,13,2147483647,12,4,1,9],"avail_no_missing":["11(0)","1(5)","4(4)","9(6)","12(3)","13(1)"],"object_location_counts":[{"shards":"1(5),4(4),9(6),11(0),12(3),13(1)","objects":8}],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"107'40","reported_seq":157,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.858930+0000","last_change":"2024-09-16T06:49:32.835071+0000","last_active":"2024-09-16T06:49:34.858930+0000","last_peered":"2024-09-16T06:49:34.858930+0000","last_clean":"2024-09-16T06:49:34.858930+0000","last_became_active":"2024-09-16T06:49:05.925903+0000","last_became_peered":"2024-09-16T06:49:05.925903+0000","last_unstale":"2024-09-16T06:49:34.858930+0000","last_undegraded":"2024-09-16T06:49:34.858930+0000","last_fullsized":"2024-09-16T06:49:34.858930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'31","last_scrub_stamp":"2024-09-16T06:49:25.350164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:25.350164+0000","objects_scrubbed":3,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:43.192516+0000","scrub_duration":0.0082238669999999993,"objects_trimmed":0,"snaptrim_duration":0.00123594,"stat_sum":{"num_bytes":409600,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":1515,"num_write":28,"num_write_kb":496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,1,10,3,8,7,14],"acting":[13,1,10,3,8,7,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.a","version":"105'244","reported_seq":505,"reported_epoch":106,"state":"peering","last_fresh":"2024-09-16T06:49:33.830702+0000","last_change":"2024-09-16T06:49:33.831281+0000","last_active":"2024-09-16T06:49:33.830702+0000","last_peered":"2024-09-16T06:49:33.034204+0000","last_clean":"2024-09-16T06:49:30.014813+0000","last_became_active":"2024-09-16T06:49:32.858571+0000","last_became_peered":"2024-09-16T06:49:32.858571+0000","last_unstale":"2024-09-16T06:49:33.830702+0000","last_undegraded":"2024-09-16T06:49:33.830702+0000","last_fullsized":"2024-09-16T06:49:33.830702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"98'224","last_scrub_stamp":"2024-09-16T06:49:26.790816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.790816+0000","objects_scrubbed":14,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:46.325635+0000","scrub_duration":0.44054632900000001,"objects_trimmed":0,"snaptrim_duration":0.040880347999999997,"stat_sum":{"num_bytes":24035328,"num_objects":16,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":41,"num_read_kb":13839,"num_write":60,"num_write_kb":24336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":31260672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,9,8,15,11],"acting":[3,7,6,9,8,15,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6,7,8,9,10,11,15],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.9","version":"105'60","reported_seq":302,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.835957+0000","last_change":"2024-09-16T06:49:32.844251+0000","last_active":"2024-09-16T06:49:34.835957+0000","last_peered":"2024-09-16T06:49:34.835957+0000","last_clean":"2024-09-16T06:49:34.835957+0000","last_became_active":"2024-09-16T06:49:31.031566+0000","last_became_peered":"2024-09-16T06:49:31.031566+0000","last_unstale":"2024-09-16T06:49:34.835957+0000","last_undegraded":"2024-09-16T06:49:34.835957+0000","last_fullsized":"2024-09-16T06:49:34.835957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:48.756986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013473433,"stat_sum":{"num_bytes":8454144,"num_objects":4,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":18150,"num_write":22,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":17514496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,2,13,4,15,9,1],"acting":[12,2,13,4,15,9,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.8","version":"105'45","reported_seq":238,"reported_epoch":107,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:34.834040+0000","last_change":"2024-09-16T06:49:25.247568+0000","last_active":"2024-09-16T06:49:34.834040+0000","last_peered":"2024-09-16T06:49:34.834040+0000","last_clean":"2024-09-16T06:49:09.868266+0000","last_became_active":"2024-09-16T06:49:25.217434+0000","last_became_peered":"2024-09-16T06:49:25.217434+0000","last_unstale":"2024-09-16T06:49:34.834040+0000","last_undegraded":"2024-09-16T06:49:24.957956+0000","last_fullsized":"2024-09-16T06:49:24.957445+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:45.535736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015152634999999999,"stat_sum":{"num_bytes":5472256,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":8,"num_read_kb":4,"num_write":55,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":4472832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0,4,2147483647,14,2],"acting":[6,1,0,4,2147483647,14,2],"avail_no_missing":["6(0)","0(2)","1(1)","2(6)","4(3)","14(5)","15(5)"],"object_location_counts":[{"shards":"0(2),1(1),2(6),4(3),6(0),14(5)","objects":7}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"28'157","reported_seq":300,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.834279+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:49:34.834279+0000","last_peered":"2024-09-16T06:49:34.834279+0000","last_clean":"2024-09-16T06:49:34.834279+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:49:34.834279+0000","last_undegraded":"2024-09-16T06:49:34.834279+0000","last_fullsized":"2024-09-16T06:49:34.834279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":90,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.017064+0000","last_change":"2024-09-16T06:48:12.637310+0000","last_active":"2024-09-16T06:49:31.017064+0000","last_peered":"2024-09-16T06:49:31.017064+0000","last_clean":"2024-09-16T06:49:31.017064+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:49:31.017064+0000","last_undegraded":"2024-09-16T06:49:31.017064+0000","last_fullsized":"2024-09-16T06:49:31.017064+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.2","version":"101'195","reported_seq":382,"reported_epoch":104,"state":"remapped+peering","last_fresh":"2024-09-16T06:49:31.830843+0000","last_change":"2024-09-16T06:49:31.831278+0000","last_active":"2024-09-16T06:49:30.009235+0000","last_peered":"2024-09-16T06:49:29.640414+0000","last_clean":"2024-09-16T06:49:29.640414+0000","last_became_active":"2024-09-16T06:48:20.888283+0000","last_became_peered":"2024-09-16T06:48:20.888283+0000","last_unstale":"2024-09-16T06:49:31.830843+0000","last_undegraded":"2024-09-16T06:49:31.830843+0000","last_fullsized":"2024-09-16T06:49:31.830843+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:36.741864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.02739635,"stat_sum":{"num_bytes":12500992,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":29,"num_read_kb":6281,"num_write":133,"num_write_kb":12032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,12,3,4,15,7,0],"acting":[14,12,3,4,15,2147483647,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,4,7,10,11,12,15],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"15","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":103,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T06:49:32.832704+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:49:32.832704+0000","last_peered":"2024-09-16T06:49:32.832704+0000","last_clean":"2024-09-16T06:49:32.832704+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:49:32.832704+0000","last_undegraded":"2024-09-16T06:49:32.832704+0000","last_fullsized":"2024-09-16T06:49:32.832704+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"107'92","reported_seq":178,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.856764+0000","last_change":"2024-09-16T06:49:32.861640+0000","last_active":"2024-09-16T06:49:34.856764+0000","last_peered":"2024-09-16T06:49:34.856764+0000","last_clean":"2024-09-16T06:49:34.856764+0000","last_became_active":"2024-09-16T06:49:05.936871+0000","last_became_peered":"2024-09-16T06:49:05.936871+0000","last_unstale":"2024-09-16T06:49:34.856764+0000","last_undegraded":"2024-09-16T06:49:34.856764+0000","last_fullsized":"2024-09-16T06:49:34.856764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'75","last_scrub_stamp":"2024-09-16T06:49:24.392872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.392872+0000","objects_scrubbed":5,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:49.469666+0000","scrub_duration":0.0091732289999999998,"objects_trimmed":0,"snaptrim_duration":0.027918088000000001,"stat_sum":{"num_bytes":5865472,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":17,"num_read_kb":16711,"num_write":30,"num_write_kb":6320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5292032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,11,14,10,12,0,2],"acting":[13,11,14,10,12,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.017107+0000","last_change":"2024-09-16T06:48:12.637304+0000","last_active":"2024-09-16T06:49:31.017107+0000","last_peered":"2024-09-16T06:49:31.017107+0000","last_clean":"2024-09-16T06:49:31.017107+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:49:31.017107+0000","last_undegraded":"2024-09-16T06:49:31.017107+0000","last_fullsized":"2024-09-16T06:49:31.017107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.0","version":"107'67","reported_seq":278,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T06:49:34.876206+0000","last_change":"2024-09-16T06:49:32.831990+0000","last_active":"2024-09-16T06:49:34.876206+0000","last_peered":"2024-09-16T06:49:34.876206+0000","last_clean":"2024-09-16T06:49:34.876206+0000","last_became_active":"2024-09-16T06:48:21.124782+0000","last_became_peered":"2024-09-16T06:48:21.124782+0000","last_unstale":"2024-09-16T06:49:34.876206+0000","last_undegraded":"2024-09-16T06:49:34.876206+0000","last_fullsized":"2024-09-16T06:49:34.876206+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.918920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013075929999999999,"stat_sum":{"num_bytes":5079040,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":775,"num_write":23,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,11,15,2,6,14,9],"acting":[12,11,15,2,6,14,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.2","version":"28'2","reported_seq":94,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T06:49:32.840680+0000","last_change":"2024-09-16T06:48:11.768932+0000","last_active":"2024-09-16T06:49:32.840680+0000","last_peered":"2024-09-16T06:49:32.840680+0000","last_clean":"2024-09-16T06:49:32.840680+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:49:32.840680+0000","last_undegraded":"2024-09-16T06:49:32.840680+0000","last_fullsized":"2024-09-16T06:49:32.840680+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"104'158","reported_seq":376,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.880654+0000","last_change":"2024-09-16T06:49:31.859053+0000","last_active":"2024-09-16T06:49:31.880654+0000","last_peered":"2024-09-16T06:49:31.880654+0000","last_clean":"2024-09-16T06:49:31.880654+0000","last_became_active":"2024-09-16T06:48:21.121223+0000","last_became_peered":"2024-09-16T06:48:21.121223+0000","last_unstale":"2024-09-16T06:49:31.880654+0000","last_undegraded":"2024-09-16T06:49:31.880654+0000","last_fullsized":"2024-09-16T06:49:31.880654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:41.389211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022564463,"stat_sum":{"num_bytes":13238272,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":62,"num_read_kb":25509,"num_write":88,"num_write_kb":13760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,1,15,4,2,3],"acting":[10,7,1,15,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"3.5","version":"82'12","reported_seq":242,"reported_epoch":104,"state":"active+undersized+degraded","last_fresh":"2024-09-16T06:49:31.884037+0000","last_change":"2024-09-16T06:49:31.885499+0000","last_active":"2024-09-16T06:49:31.884037+0000","last_peered":"2024-09-16T06:49:31.884037+0000","last_clean":"2024-09-16T06:49:09.867185+0000","last_became_active":"2024-09-16T06:49:31.848579+0000","last_became_peered":"2024-09-16T06:49:31.848579+0000","last_unstale":"2024-09-16T06:49:31.884037+0000","last_undegraded":"2024-09-16T06:49:31.837321+0000","last_fullsized":"2024-09-16T06:49:31.835689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017739370000000001,"stat_sum":{"num_bytes":2555904,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":17765,"num_write":7,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13,2147483647,2,7,9,8],"acting":[15,13,2147483647,2,7,9,8],"avail_no_missing":["15(0)","1(5)","2(3)","7(4)","8(6)","9(5)","13(1)"],"object_location_counts":[{"shards":"2(3),7(4),8(6),9(5),13(1),15(0)","objects":2}],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.024656+0000","last_change":"2024-09-16T06:48:12.417890+0000","last_active":"2024-09-16T06:49:31.024656+0000","last_peered":"2024-09-16T06:49:31.024656+0000","last_clean":"2024-09-16T06:49:31.024656+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:49:31.024656+0000","last_undegraded":"2024-09-16T06:49:31.024656+0000","last_fullsized":"2024-09-16T06:49:31.024656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"104'185","reported_seq":446,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.877309+0000","last_change":"2024-09-16T06:49:31.853957+0000","last_active":"2024-09-16T06:49:31.877309+0000","last_peered":"2024-09-16T06:49:31.877309+0000","last_clean":"2024-09-16T06:49:31.877309+0000","last_became_active":"2024-09-16T06:49:05.920150+0000","last_became_peered":"2024-09-16T06:49:05.920150+0000","last_unstale":"2024-09-16T06:49:31.877309+0000","last_undegraded":"2024-09-16T06:49:31.877309+0000","last_fullsized":"2024-09-16T06:49:31.877309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:38.017254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.022284939,"stat_sum":{"num_bytes":7028736,"num_objects":10,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":41,"num_read_kb":30973,"num_write":74,"num_write_kb":8560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":9486336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,12,9,2,6,10,13],"acting":[1,12,9,2,6,10,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+undersized","last_fresh":"2024-09-16T06:49:34.837869+0000","last_change":"2024-09-16T06:49:11.861382+0000","last_active":"2024-09-16T06:49:34.837869+0000","last_peered":"2024-09-16T06:49:34.837869+0000","last_clean":"2024-09-16T06:49:09.866035+0000","last_became_active":"2024-09-16T06:49:11.861382+0000","last_became_peered":"2024-09-16T06:49:11.861382+0000","last_unstale":"2024-09-16T06:49:34.837869+0000","last_undegraded":"2024-09-16T06:49:34.837869+0000","last_fullsized":"2024-09-16T06:49:11.860463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8],"acting":[8],"avail_no_missing":["8"],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"106'169","reported_seq":207,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-16T06:49:33.902387+0000","last_change":"2024-09-16T06:49:32.872905+0000","last_active":"2024-09-16T06:49:33.902387+0000","last_peered":"2024-09-16T06:49:33.902387+0000","last_clean":"2024-09-16T06:49:33.902387+0000","last_became_active":"2024-09-16T06:49:05.952263+0000","last_became_peered":"2024-09-16T06:49:05.952263+0000","last_unstale":"2024-09-16T06:49:33.902387+0000","last_undegraded":"2024-09-16T06:49:33.902387+0000","last_fullsized":"2024-09-16T06:49:33.902387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038875240999999998,"stat_sum":{"num_bytes":17760256,"num_objects":13,"num_object_clones":9,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":44,"num_read_kb":47767,"num_write":100,"num_write_kb":19216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":34914304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,6,1,14,10,11],"acting":[13,8,6,1,14,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":89,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.025403+0000","last_change":"2024-09-16T06:48:11.775015+0000","last_active":"2024-09-16T06:49:31.025403+0000","last_peered":"2024-09-16T06:49:31.025403+0000","last_clean":"2024-09-16T06:49:31.025403+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:49:31.025403+0000","last_undegraded":"2024-09-16T06:49:31.025403+0000","last_fullsized":"2024-09-16T06:49:31.025403+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:48:08.638478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:49:35.779513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.6","version":"105'39","reported_seq":44,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-16T06:49:33.829589+0000","last_change":"2024-09-16T06:49:32.844621+0000","last_active":"2024-09-16T06:49:33.829589+0000","last_peered":"2024-09-16T06:49:33.829589+0000","last_clean":"2024-09-16T06:49:33.829589+0000","last_became_active":"2024-09-16T06:49:31.036738+0000","last_became_peered":"2024-09-16T06:49:31.036738+0000","last_unstale":"2024-09-16T06:49:33.829589+0000","last_undegraded":"2024-09-16T06:49:33.829589+0000","last_fullsized":"2024-09-16T06:49:33.829589+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:19.848202+0000","last_clean_scrub_stamp":"2024-09-16T06:48:19.848202+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011710752,"stat_sum":{"num_bytes":2932736,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":15176,"num_write":9,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":6307840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,14,6,2,1,7,0],"acting":[3,14,6,2,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"2"},{"start":"1f","length":"3"},{"start":"29","length":"4"},{"start":"2e","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":102,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T06:49:31.017805+0000","last_change":"2024-09-16T06:49:12.584897+0000","last_active":"2024-09-16T06:49:31.017805+0000","last_peered":"2024-09-16T06:49:31.017805+0000","last_clean":"2024-09-16T06:49:31.017805+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:49:31.017805+0000","last_undegraded":"2024-09-16T06:49:31.017805+0000","last_fullsized":"2024-09-16T06:49:31.017805+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:12.584669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:12.584669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:13.936208+0000","scrub_duration":0.0037478810000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":146440192,"num_objects":127,"num_object_clones":80,"num_object_copies":889,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":9,"num_read":441,"num_read_kb":283600,"num_write":802,"num_write_kb":154384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":160563200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258154496,"data_stored":408563712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1724,"ondisk_log_size":1724,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":15,"acting":15,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247827,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46724,"kb_used_data":15604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325116,"statfs":{"total":96636764160,"available":96588918784,"internally_reserved":0,"allocated":15978496,"data_stored":23167488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":45.249000000000002,"5min":45.249000000000002,"15min":45.249000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":780.85000000000002,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":9.7080000000000002,"5min":9.7080000000000002,"15min":9.7080000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":164.398,"5min":164.398,"15min":164.398},"last":1.0349999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.113}]},{"osd":2,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.88500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":1.238},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.21}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":13.01,"5min":13.01,"15min":13.01},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":225.697,"5min":225.697,"15min":225.697},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.0900000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.73699999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":4.4379999999999997,"5min":4.4379999999999997,"15min":4.4379999999999997},"last":348.30599999999998},{"interface":"front","average":{"1min":12.073,"5min":12.073,"15min":12.073},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":197.559,"5min":197.559,"15min":197.559},"last":1.3979999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.2689999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1919999999999999}]}]},{"osd":14,"up_from":23,"seq":98784247829,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47320,"kb_used_data":16196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324520,"statfs":{"total":96636764160,"available":96588308480,"internally_reserved":0,"allocated":16584704,"data_stored":24910762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":27.170999999999999,"5min":27.170999999999999,"15min":27.170999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":525.66800000000001,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":105.913}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.242}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.272}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.95399999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.333},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.226}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.151}]},{"osd":12,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.464}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.552},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.3640000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":360.42200000000003,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.413},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":3.7709999999999999,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.526}]}]},{"osd":11,"up_from":23,"seq":98784247828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70676,"kb_used_data":21504,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94301164,"statfs":{"total":96636764160,"available":96564391936,"internally_reserved":0,"allocated":22020096,"data_stored":36871082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":50.844000000000001,"5min":50.844000000000001,"15min":50.844000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":949.71199999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":48.140999999999998,"5min":48.140999999999998,"15min":48.140999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":894.49900000000002,"5min":894.49900000000002,"15min":894.49900000000002},"last":0.84999999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.163},{"interface":"front","average":{"1min":4.8739999999999997,"5min":4.8739999999999997,"15min":4.8739999999999997},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":71.503,"5min":71.503,"15min":71.503},"last":0.95899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.879},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.7400000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.139}]},{"osd":10,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.6469999999999998,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.006},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.1100000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.044},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.1830000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.274},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.298}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313237,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48408,"kb_used_data":17288,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323432,"statfs":{"total":96636764160,"available":96587194368,"internally_reserved":0,"allocated":17702912,"data_stored":24518377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,6,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3660000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.298},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1819999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":2.052},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":2.0699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.113},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.86399999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.73399999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1219999999999999,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.0620000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":42.863,"5min":42.863,"15min":42.863},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":502.57400000000001,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":41.012,"5min":41.012,"15min":41.012},"min":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"max":{"1min":502.62200000000001,"5min":502.62200000000001,"15min":502.62200000000001},"last":0.95199999999999996}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.393}]},{"osd":12,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.347},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.234}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.2869999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313237,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":29508,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":30216192,"data_stored":39507882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":8.6449999999999996,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.96799999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":28.696999999999999,"5min":28.696999999999999,"15min":28.696999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":385.83800000000002,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.167}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.90800000000000003}]},{"osd":8,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.68300000000000005},{"interface":"front","average":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":96.432000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.0800000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.2330000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":10.728,"5min":10.728,"15min":10.728},"last":1.254},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":3.8069999999999999,"5min":3.8069999999999999,"15min":3.8069999999999999},"last":1.1859999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.3180000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2,"5min":2,"15min":2},"last":1.379}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39748,"kb_used_data":8632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332092,"statfs":{"total":96636764160,"available":96596062208,"internally_reserved":0,"allocated":8839168,"data_stored":11416608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":39.485999999999997,"5min":39.485999999999997,"15min":39.485999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":734.39200000000005,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.113},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.96599999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.0149999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.754},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":32.578000000000003,"5min":32.578000000000003,"15min":32.578000000000003},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":596.70600000000002,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.98399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.079},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":0.89100000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.3119999999999998,"5min":2.3119999999999998,"15min":2.3119999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.7149999999999999,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":1.0620000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.169},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.036}]}]},{"osd":4,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49304,"kb_used_data":18188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94322536,"statfs":{"total":96636764160,"available":96586276864,"internally_reserved":0,"allocated":18624512,"data_stored":30310126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.96799999999999997},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":14.775,"5min":14.775,"15min":14.775},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":265.48099999999999,"5min":265.48099999999999,"15min":265.48099999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.022}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.133},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":1.1080000000000001}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":31.298999999999999,"5min":31.298999999999999,"15min":31.298999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":576.29200000000003,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.038}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.755},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.83399999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.99,"15min":1.99},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":20.283999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":16.338999999999999,"5min":16.338999999999999,"15min":16.338999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":306.85599999999999,"5min":306.85599999999999,"15min":306.85599999999999},"last":0.94199999999999995}]},{"osd":10,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.2390000000000001},{"interface":"front","average":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":21.588999999999999,"5min":21.588999999999999,"15min":21.588999999999999},"last":1.196}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.337}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.226},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.119}]},{"osd":15,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.306},{"interface":"front","average":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":20.437999999999999,"5min":20.437999999999999,"15min":20.437999999999999},"last":1.246}]}]},{"osd":5,"up_from":22,"seq":94489280527,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34536,"kb_used_data":7508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337304,"statfs":{"total":96636764160,"available":96601399296,"internally_reserved":0,"allocated":7688192,"data_stored":10847401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,8,10,11,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1579999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]}]},{"osd":6,"up_from":22,"seq":94489280533,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49308,"kb_used_data":18196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94322532,"statfs":{"total":96636764160,"available":96586272768,"internally_reserved":0,"allocated":18632704,"data_stored":28162506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,7,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.6000000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.74}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.0109999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.4890000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":13.279,"5min":13.279,"15min":13.279},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":231.90700000000001,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.6859999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":2.0470000000000002},{"interface":"front","average":{"1min":18.225000000000001,"5min":18.225000000000001,"15min":18.225000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":323.55500000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.4510000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.84999999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.073},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.0920000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":2.1829999999999998},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.5760000000000001}]}]},{"osd":7,"up_from":22,"seq":94489280535,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49560,"kb_used_data":18436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94322280,"statfs":{"total":96636764160,"available":96586014720,"internally_reserved":0,"allocated":18878464,"data_stored":24911598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.59699999999999998},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.98199999999999998},{"interface":"front","average":{"1min":29.103999999999999,"5min":29.103999999999999,"15min":29.103999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":527.10299999999995,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.878}]},{"osd":2,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":6.1749999999999998,"5min":6.1749999999999998,"15min":6.1749999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":136.72399999999999,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.91100000000000003}]},{"osd":3,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":20.370999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":508.52100000000002,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.012}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":30.073,"5min":30.073,"15min":30.073},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":415.30099999999999,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":37.726999999999997,"5min":37.726999999999997,"15min":37.726999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":502.55200000000002,"5min":502.55200000000002,"15min":502.55200000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":20.564,"5min":20.564,"15min":20.564},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":502.39600000000002,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.0329999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":12,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.05},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.1220000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.103},{"interface":"front","average":{"1min":21.131,"5min":21.131,"15min":21.131},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":514.07799999999997,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.2010000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44532,"kb_used_data":13408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327308,"statfs":{"total":96636764160,"available":96591163392,"internally_reserved":0,"allocated":13729792,"data_stored":22023918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.008},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.38900000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":10.706,"5min":10.706,"15min":10.706},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":177.92400000000001,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.754},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.70599999999999996}]},{"osd":6,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":27.687000000000001,"5min":27.687000000000001,"15min":27.687000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":449.88,"5min":449.88,"15min":449.88},"last":0.499},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.1359999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.59799999999999998}]},{"osd":9,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.903,"5min":1.903,"15min":1.903},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":12.366,"5min":12.366,"15min":12.366},"last":1.3080000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:14 2024","interfaces":[{"interface":"back","average":{"1min":56.750999999999998,"5min":56.750999999999998,"15min":56.750999999999998},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":998.42399999999998,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.331}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":0.77000000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:49:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.5869999999999997,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.3200000000000001}]}]},{"osd":12,"up_from":22,"seq":94489280534,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":20424,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":20914176,"data_stored":33300206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":0.64900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90000000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.4119999999999999,"5min":3.4119999999999999,"15min":3.4119999999999999},"last":151.34800000000001},{"interface":"front","average":{"1min":38.109000000000002,"5min":38.109000000000002,"15min":38.109000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":708.49699999999996,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.627}]},{"osd":4,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":33.972000000000001,"5min":33.972000000000001,"15min":33.972000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":630.75900000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.72099999999999997},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.86599999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.3330000000000002,"5min":4.3330000000000002,"15min":4.3330000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.95399999999999996}]},{"osd":11,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":50.134999999999998,"5min":50.134999999999998,"15min":50.134999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":980.38300000000004,"5min":980.38300000000004,"15min":980.38300000000004},"last":150.93299999999999}]},{"osd":13,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":229.631}]},{"osd":14,"last update":"Mon Sep 16 06:49:12 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.997},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.91900000000000004}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.93100000000000005}]}]},{"osd":1,"up_from":23,"seq":98784247830,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53044,"kb_used_data":21920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94318796,"statfs":{"total":96636764160,"available":96582447104,"internally_reserved":0,"allocated":22446080,"data_stored":37641130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":12.209,"5min":12.209,"15min":12.209},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":214.93299999999999,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.361},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":12.398,"5min":12.398,"15min":12.398},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":216.958,"5min":216.958,"15min":216.958},"last":1.036},{"interface":"front","average":{"1min":35.026000000000003,"5min":35.026000000000003,"15min":35.026000000000003},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":645.88099999999997,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.68500000000000005}]},{"osd":9,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":28.591000000000001,"5min":28.591000000000001,"15min":28.591000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":493.76100000000002,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.871},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3100000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.2809999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":37.526000000000003,"5min":37.526000000000003,"15min":37.526000000000003},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":687.10799999999995,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.258},{"interface":"front","average":{"1min":26.273,"5min":26.273,"15min":26.273},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":475.75700000000001,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.103},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.1970000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.3919999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.45},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.3380000000000001}]}]},{"osd":2,"up_from":23,"seq":98784247828,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46768,"kb_used_data":15648,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325072,"statfs":{"total":96636764160,"available":96588873728,"internally_reserved":0,"allocated":16023552,"data_stored":22241792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.879},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.72499999999999998}]},{"osd":3,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.82399999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":19.053000000000001,"5min":19.053000000000001,"15min":19.053000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":361.36799999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.38200000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":35.137999999999998,"5min":35.137999999999998,"15min":35.137999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":646.98800000000006,"5min":646.98800000000006,"15min":646.98800000000006},"last":0.64100000000000001},{"interface":"front","average":{"1min":28.411999999999999,"5min":28.411999999999999,"15min":28.411999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":516.49000000000001,"5min":516.49000000000001,"15min":516.49000000000001},"last":0.89900000000000002}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":6.2279999999999998,"5min":6.2279999999999998,"15min":6.2279999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":3.8239999999999998,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.66200000000000003}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.6269999999999998,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":0.86499999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":9,"up_from":23,"seq":98784247827,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44212,"kb_used_data":13092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327628,"statfs":{"total":96636764160,"available":96591491072,"internally_reserved":0,"allocated":13406208,"data_stored":20861459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":50.427999999999997,"5min":50.427999999999997,"15min":50.427999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":941.91899999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.71399999999999997}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":104.761,"5min":104.761,"15min":104.761},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":20.274000000000001,"5min":20.274000000000001,"15min":20.274000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":363.67399999999998,"5min":363.67399999999998,"15min":363.67399999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.98199999999999998}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.0049999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":1.018}]},{"osd":13,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.0549999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.042}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":49.017000000000003,"5min":49.017000000000003,"15min":49.017000000000003},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":750.83299999999997,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.077}]}]},{"osd":10,"up_from":23,"seq":98784247828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67160,"kb_used_data":17988,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94304680,"statfs":{"total":96636764160,"available":96567992320,"internally_reserved":0,"allocated":18419712,"data_stored":28482474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[1,2,3,4,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.93200000000000005},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":599.428},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.232}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.64900000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.071},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.1930000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.153}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.101}]},{"osd":9,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.198},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.99399999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.3049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6787072,"data_stored":10506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20905984,"data_stored":36741120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14659584,"data_stored":21512192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":25702400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17018880,"data_stored":29368320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7049216,"data_stored":10805248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16510976,"data_stored":26701824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":17272832,"data_stored":23969792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12124160,"data_stored":21082112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":12652544,"data_stored":21250048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16879616,"data_stored":27582464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480000,"data_stored":35971072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":19308544,"data_stored":32358400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28626944,"data_stored":38576128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":15044608,"data_stored":24010752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":14602240,"data_stored":22425600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:49:37.637 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-16T06:49:37.637 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-16T06:49:37.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-16T06:49:37.638 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:37.638 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:37.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T06:49:37.676 INFO:tasks.ceph.osd.10.smithi115.stderr:2024-09-16T06:49:37.674+0000 7f4d617f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26182) UID: 0 2024-09-16T06:49:37.762 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:37.762 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:37.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T06:49:37.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-16T06:49:37.777 INFO:tasks.ceph.osd.14.smithi115.stderr:2024-09-16T06:49:37.774+0000 7fd4e1109640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26184) UID: 0 2024-09-16T06:49:37.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:49:37.877 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:37.874+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:37.908 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-16T06:49:37.915 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-16T06:49:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T06:49:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-16T06:49:37.936 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T06:49:37.960 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-16T06:49:37.961 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T06:49:37.977 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-16T06:49:37.977+0000 7fd5bb5ea640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26167) UID: 0 2024-09-16T06:49:37.985 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-16T06:49:37.985 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T06:49:38.006 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-16T06:49:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T06:49:38.007 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-16T06:49:38.031 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-16T06:49:38.031 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T06:49:38.032 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-16T06:49:38.032 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T06:49:38.047 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-16T06:49:38.048 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'unique_pool_0' removed 2024-09-16T06:49:38.064 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-16T06:49:38.064 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T06:49:38.078 INFO:tasks.ceph.osd.11.smithi117.stderr:2024-09-16T06:49:38.078+0000 7fb8833d5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26549) UID: 0 2024-09-16T06:49:38.079 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T06:49:38.087 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T06:49:38.087 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8: {} 2024-09-16T06:49:38.088 INFO:teuthology.orchestra.run.smithi100.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T06:49:38.107 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9: {} 2024-09-16T06:49:38.107 INFO:teuthology.orchestra.run.smithi100.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T06:49:38.131 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10: {} 2024-09-16T06:49:38.131 INFO:teuthology.orchestra.run.smithi100.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T06:49:38.157 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11: {} 2024-09-16T06:49:38.157 INFO:teuthology.orchestra.run.smithi100.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T06:49:38.181 INFO:teuthology.orchestra.run.smithi100.stdout:osd.12: {} 2024-09-16T06:49:38.181 INFO:teuthology.orchestra.run.smithi100.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-16T06:49:38.208 INFO:teuthology.orchestra.run.smithi100.stdout:osd.13: {} 2024-09-16T06:49:38.208 INFO:teuthology.orchestra.run.smithi100.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:9 3.1s0 3:825f7e25:::smithi11731613-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.323253049000002, 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.040864080999999997, 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.222 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 9 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896610+0000", 2024-09-16T06:49:38.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896609+0000", 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896617+0000", 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9589999999999995e-06 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896634+0000", 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6872e-05 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.896647+0000", 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3621999999999999e-05 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.902794+0000", 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061462820000000003 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.903015+0000", 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221487 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.928840+0000", 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025824686999999999 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.929046+0000", 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020604299999999999 2024-09-16T06:49:38.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.937474+0000", 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0084285460000000003 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:17 3.bs0 3:db3fccb6:::smithi11731613-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.321976410999994, 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034791753000000002, 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 17 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897887+0000", 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897885+0000", 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897900+0000", 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5515e-05 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897935+0000", 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4703000000000003e-05 2024-09-16T06:49:38.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.897965+0000", 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9884000000000001e-05 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905007+0000", 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007042296 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905241+0000", 2024-09-16T06:49:38.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023388 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.906656+0000", 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014152209999999999 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.912065+0000", 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054085640000000003 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932510+0000", 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.020444922000000001 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932679+0000", 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168894 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:18 3.bs0 3:db3fccb6:::smithi11731613-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:38.233 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.321817874000004, 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.041379560000000003, 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 18 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898046+0000", 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898044+0000", 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898055+0000", 2024-09-16T06:49:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0902e-05 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898077+0000", 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2395000000000002e-05 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.898095+0000", 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7825e-05 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907329+0000", 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092342250000000004 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907544+0000", 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021447400000000001 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.932831+0000", 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025286896999999999 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.933015+0000", 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018463 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939425+0000", 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064098250000000001 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:38 3.bs0 3:d4957fab:::smithi11731613-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.314166182999998, 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034120408999999997, 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 38 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.239 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905697+0000", 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905696+0000", 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905706+0000", 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9890000000000004e-06 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905725+0000", 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9341e-05 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905739+0000", 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4336e-05 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907725+0000", 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001985849 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907741+0000", 2024-09-16T06:49:38.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5401e-05 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918115+0000", 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010374431999999999 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.918349+0000", 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023440900000000001 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.919836+0000", 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014862639999999999 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.925093+0000", 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052577300000000004 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939675+0000", 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014581904999999999 2024-09-16T06:49:38.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.939818+0000", 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014229600000000001 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:39 3.bs0 3:d4957fab:::smithi11731613-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.314110055, 2024-09-16T06:49:38.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039069524000000001, 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 39 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905753+0000", 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905752+0000", 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905759+0000", 2024-09-16T06:49:38.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905776+0000", 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6694e-05 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.905788+0000", 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2336e-05 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907813+0000", 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020251790000000002 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.907828+0000", 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5094e-05 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.921873+0000", 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014044235 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.922092+0000", 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000219514 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940712+0000", 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018619988000000001 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.940891+0000", 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017858199999999999 2024-09-16T06:49:38.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:22.944823+0000", 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039321989999999999 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:152 3.7s0 3:ebfb55a4:::smithi11731613-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:38.251 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.143455966999994, 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44514418, 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 152 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076407+0000", 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076406+0000", 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076414+0000", 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2889999999999998e-06 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076425+0000", 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0671e-05 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076433+0000", 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0870000000000007e-06 2024-09-16T06:49:38.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079120+0000", 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026864800000000002 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079194+0000", 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4874000000000002e-05 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.079523+0000", 2024-09-16T06:49:38.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032904499999999998 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080194+0000", 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00067098400000000001 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521344+0000", 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.441149183 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.521552+0000", 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020799600000000001 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:153 3.7s0 3:ebfb55a4:::smithi11731613-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.143422141000002, 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44943085999999999, 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 153 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076441+0000", 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076440+0000", 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076444+0000", 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0559999999999998e-06 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076455+0000", 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0789e-05 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.076466+0000", 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0241000000000001e-05 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080491+0000", 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040256060000000001 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.080588+0000", 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6849999999999996e-05 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522277+0000", 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.44168930299999998 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.522519+0000", 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241246 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:48:23.525872+0000", 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033535660000000001 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:695 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~4194304,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.001438+0000", 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.218425709000002, 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.036045328000000001, 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 695 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001438+0000", 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001438+0000", 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001437+0000", 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001440+0000", 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1e-06 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001445+0000", 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8620000000000002e-06 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001449+0000", 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4790000000000001e-06 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003572+0000", 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021228150000000001 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-16T06:49:38.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003819+0000", 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024764200000000001 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.020118+0000", 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016299088 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.020504+0000", 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038530599999999997 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.037483+0000", 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016979448000000001 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:696 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~4194304,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.218384153000002, 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038928996, 2024-09-16T06:49:38.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 696 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001479+0000", 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001481+0000", 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3700000000000002e-06 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001486+0000", 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2359999999999998e-06 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.001489+0000", 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4199999999999999e-06 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.003913+0000", 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024241890000000002 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.004188+0000", 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000274906 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.025189+0000", 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021001427 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.025394+0000", 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000204334 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:06.040408+0000", 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015014394 2024-09-16T06:49:38.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:741 3.bs0 3:d4957fab:::smithi11731613-13:28 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:09.846644+0000", 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.373219472999999, 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.036566170000000002, 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 741 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846644+0000", 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846644+0000", 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846642+0000", 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846652+0000", 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3729999999999999e-06 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846672+0000", 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0553000000000001e-05 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.846687+0000", 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5063e-05 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.847038+0000", 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035066999999999997 2024-09-16T06:49:38.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.847053+0000", 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.448e-05 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.855965+0000", 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089126729999999994 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.856204+0000", 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023855099999999999 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:09.883210+0000", 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.027006220000000001 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.279 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "MOSDECSubOpWriteReply(3.7s0 98/77 ECSubWriteReply(tid=195, last_complete=98'146, committed=1, applied=1))", 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:26.001343+0000", 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.218519917, 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.93021421400000004, 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.001343+0000", 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.001343+0000", 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.001342+0000", 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.001351+0000", 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.089e-06 2024-09-16T06:49:38.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.930364+0000", 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.92901333699999999 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.930377+0000", 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3276e-05 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.930542+0000", 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000165131 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:26.931558+0000", 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001015398 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:949 3.7s0 3:e60b6e1b:::smithi11731613-14:head [copy-from ver 19] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:38.283 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:32.835788+0000", 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3840754789999998, 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048337106999999997, 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 949 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.835788+0000", 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.835788+0000", 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.835785+0000", 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.835804+0000", 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8609000000000001e-05 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.835830+0000", 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6013e-05 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.835849+0000", 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9766e-05 2024-09-16T06:49:38.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.839597+0000", 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037474380000000001 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.839858+0000", 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026070200000000003 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.866206+0000", 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.026348376999999999 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.871993+0000", 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057869840000000002 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.883905+0000", 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011912124 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.884125+0000", 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000219967 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:978 3.7s0 3:e8f80969:::smithi11731613-49:head [setxattr _header (57) in=7b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:32.906458+0000", 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.313405189, 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.65705895999999997, 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 978 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906458+0000", 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906458+0000", 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906457+0000", 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906464+0000", 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6579999999999996e-06 2024-09-16T06:49:38.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906473+0000", 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0520000000000007e-06 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906481+0000", 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8720000000000002e-06 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.910413+0000", 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039317969999999999 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.910632+0000", 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021980599999999999 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.911452+0000", 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00081918500000000005 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.912369+0000", 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00091748399999999999 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.563352+0000", 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.65098246699999995 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.563517+0000", 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000165575 2024-09-16T06:49:38.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:979 3.7s0 3:e8f80969:::smithi11731613-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:32.906522+0000", 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3133413860000003, 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.66076030200000002, 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 979 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906522+0000", 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906522+0000", 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906521+0000", 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906525+0000", 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7000000000000002e-06 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906537+0000", 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2084e-05 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.906544+0000", 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5180000000000001e-06 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.912013+0000", 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054689279999999996 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:32.912139+0000", 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126301 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.564376+0000", 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.65223753600000001 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.564677+0000", 2024-09-16T06:49:38.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030056100000000002 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.567282+0000", 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026052689999999999 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:984 3.7s0 3:e6e9cc11:::smithi11731613-37:41 [read 0~9224192,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:33.833228+0000", 2024-09-16T06:49:38.300 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3866355400000003, 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.038441703000000001, 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 984 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833228+0000", 2024-09-16T06:49:38.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833228+0000", 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833224+0000", 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833241+0000", 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6883000000000001e-05 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833274+0000", 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2489e-05 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833290+0000", 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6073e-05 2024-09-16T06:49:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833440+0000", 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000150202 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op must wait for map", 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833466+0000", 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5473000000000002e-05 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.837873+0000", 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004407149 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.838177+0000", 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030403400000000001 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.871670+0000", 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.033492743999999998 2024-09-16T06:49:38.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:985 3.7s0 3:e6e9cc11:::smithi11731613-37:41 [read 0~9224192,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:33.833312+0000", 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3865516329999998, 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047825562000000002, 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 985 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833312+0000", 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833312+0000", 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833311+0000", 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833319+0000", 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4530000000000002e-06 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833357+0000", 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7635999999999997e-05 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833370+0000", 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3196e-05 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833666+0000", 2024-09-16T06:49:38.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029585499999999998 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833686+0000", 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0744999999999999e-05 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839066+0000", 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053793640000000002 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839383+0000", 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031728399999999998 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.881137+0000", 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.041754228999999997 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:986 3.7s0 3:e6e9cc11:::smithi11731613-37:41 [read 0~9224192,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:38.311 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:33.833428+0000", 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3864355330000002, 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.055930227999999999, 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 986 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833428+0000", 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833428+0000", 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833427+0000", 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833433+0000", 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1059999999999996e-06 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833444+0000", 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1168999999999999e-05 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833454+0000", 2024-09-16T06:49:38.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2699999999999993e-06 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833771+0000", 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000317874 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.833792+0000", 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0293e-05 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840193+0000", 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064016200000000002 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840467+0000", 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027352499999999999 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.889358+0000", 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048891164000000001 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.316 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:987 3.7s0 3:ebfb55a4:::smithi11731613-50:head [append 0~131072] snapc 47=[47,44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:33.839556+0000", 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3803072350000001, 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.051410482, 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 987 2024-09-16T06:49:38.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839556+0000", 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839556+0000", 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839552+0000", 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839913+0000", 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000361059 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839948+0000", 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5068999999999999e-05 2024-09-16T06:49:38.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839970+0000", 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1866e-05 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.846922+0000", 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069515820000000004 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.847233+0000", 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031124499999999998 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.850036+0000", 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002803303 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.855060+0000", 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050237349999999997 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.890749+0000", 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.035688618999999998 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.890967+0000", 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021778699999999999 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:988 3.7s0 3:ebfb55a4:::smithi11731613-50:head [setxattr _header (57) in=7b] snapc 47=[47,44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:38.322 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:33.839992+0000", 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3798710979999997, 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.051678754, 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 988 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839992+0000", 2024-09-16T06:49:38.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839992+0000", 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.839991+0000", 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840000+0000", 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6430000000000007e-06 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840018+0000", 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7748999999999999e-05 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840033+0000", 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5157000000000001e-05 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.852505+0000", 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012471818000000001 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.852779+0000", 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000274042 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.853876+0000", 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010972180000000001 2024-09-16T06:49:38.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.855251+0000", 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001374992 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.891413+0000", 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.036162095999999998 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.891671+0000", 2024-09-16T06:49:38.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025780200000000001 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4511.0:989 3.7s0 3:ebfb55a4:::smithi11731613-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-16T06:49:33.840047+0000", 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.379816323, 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.062404017999999999, 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4511", 2024-09-16T06:49:38.328 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 989 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840047+0000", 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840047+0000", 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-16T06:49:38.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840046+0000", 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840053+0000", 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8199999999999999e-06 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840071+0000", 2024-09-16T06:49:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8076000000000001e-05 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.840085+0000", 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4552999999999999e-05 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.854766+0000", 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014681113000000001 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:38.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.855031+0000", 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026461600000000002 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.892823+0000", 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037791507000000002 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.893116+0000", 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029373799999999999 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-16T06:49:38.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-16T06:49:33.902451+0000", 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093347719999999999 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-16T06:49:38.333 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:38.334 INFO:teuthology.orchestra.run.smithi100.stdout:osd.14: {} 2024-09-16T06:49:38.335 INFO:teuthology.orchestra.run.smithi100.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-16T06:49:38.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:49:38.239+0000 7f24dbfff640 0 -- 172.21.15.100:0/837398855 >> [v2:172.21.15.117:6826/579912865,v1:172.21.15.117:6827/579912865] conn(0x7f24dc221080 msgr2=0x7f24dc223470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-16T06:49:38.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T06:49:38.465 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:38.466 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:38.469 INFO:teuthology.orchestra.run.smithi100.stdout:osd.15: {} 2024-09-16T06:49:38.469 INFO:teuthology.orchestra.run.smithi100.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-16T06:49:38.475 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T06:49:38.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T06:49:38.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T06:49:38.600 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:38.601 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:38.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=5, reqid=client.4511.0:59, at_version=35'4, trim_to=0'0, roll_forward_to=35'2))", 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.798148616999995, 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063718670000000005, 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941154+0000", 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941153+0000", 2024-09-16T06:49:38.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941187+0000", 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3837000000000001e-05 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002113+0000", 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060926368000000002 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002134+0000", 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-05 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002230+0000", 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5789000000000001e-05 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002246+0000", 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5676e-05 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004821+0000", 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002575262 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.004873+0000", 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2095000000000002e-05 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.8s6 35/32 ECSubWrite(tid=158, reqid=client.4511.0:77, at_version=35'2, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.763751579000001, 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051524863999999997, 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975551+0000", 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975550+0000", 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975589+0000", 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9523000000000001e-05 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021558+0000", 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045968459000000003 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021574+0000", 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6399999999999999e-05 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021656+0000", 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1742000000000004e-05 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.021673+0000", 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7343e-05 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027012+0000", 2024-09-16T06:49:38.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053383889999999998 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027076+0000", 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4325999999999994e-05 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.9s1 35/32 ECSubRead(tid=11, to_read={3:92c79891:::smithi11731613-34:head=0,4096,0}, subchunks={3:92c79891:::smithi11731613-34:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.727940062000002, 2024-09-16T06:49:38.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034091585000000001, 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011363+0000", 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011362+0000", 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.011374+0000", 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2490000000000001e-05 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044980+0000", 2024-09-16T06:49:38.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033605818000000003 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.044996+0000", 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5996999999999999e-05 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045095+0000", 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9290000000000007e-05 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045454+0000", 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000359038 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 35/32 ECSubWrite(tid=13, reqid=client.4511.0:128, at_version=35'6, trim_to=0'0, roll_forward_to=35'4))", 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.693711617000005, 2024-09-16T06:49:38.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623068000000002, 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045591+0000", 2024-09-16T06:49:38.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045590+0000", 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.045617+0000", 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.724e-05 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073137+0000", 2024-09-16T06:49:38.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027519538999999999 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073150+0000", 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3046e-05 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073218+0000", 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7879999999999994e-05 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.073228+0000", 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074167+0000", 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093834699999999997 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.074214+0000", 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7722000000000002e-05 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.0s3 35/32 ECSubRead(tid=19, to_read={3:043b7e8c:::smithi11731613-16:head=0,65536,0}, subchunks={3:043b7e8c:::smithi11731613-16:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.657557851000007, 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.14017474999999999, 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081745+0000", 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081744+0000", 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081755+0000", 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0812000000000001e-05 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221448+0000", 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.13969311000000001 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221465+0000", 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7030000000000001e-05 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221629+0000", 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163549 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.221920+0000", 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000291181 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.5s3 41/37 ECSubRead(tid=7, to_read={3:af79444c:::smithi11731613-29:head=0,118784,0}, subchunks={3:af79444c:::smithi11731613-29:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.775534045000001, 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29065317299999999, 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963769+0000", 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963768+0000", 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.963777+0000", 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2019999999999993e-06 2024-09-16T06:49:38.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254192+0000", 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.290415064 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254204+0000", 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1430000000000001e-05 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254309+0000", 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104763 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:29.254422+0000", 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011342500000000001 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 52/37 ECSubWrite(tid=112, reqid=client.4511.0:424, at_version=52'35, trim_to=0'0, roll_forward_to=50'33))", 2024-09-16T06:49:38.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.882774826000002, 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017061956, 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856528+0000", 2024-09-16T06:49:38.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856527+0000", 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856580+0000", 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2326e-05 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856691+0000", 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011141600000000001 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856715+0000", 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3710000000000002e-05 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856826+0000", 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011109599999999999 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.856852+0000", 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5686000000000001e-05 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873537+0000", 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016685114000000001 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:39.873590+0000", 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3242000000000003e-05 2024-09-16T06:49:38.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s3 59/32 ECSubWrite(tid=54, reqid=client.4511.0:469, at_version=59'36, trim_to=0'0, roll_forward_to=57'34))", 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.898705636000003, 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.91193373600000005, 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840597+0000", 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:46.840907+0000", 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000310045 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749945+0000", 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.90903863699999998 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.749964+0000", 2024-09-16T06:49:38.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8073000000000001e-05 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750059+0000", 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5636000000000004e-05 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.750079+0000", 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9893000000000001e-05 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752481+0000", 2024-09-16T06:49:38.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024015989999999999 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:47.752531+0000", 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0263000000000003e-05 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=90, reqid=osd.1.0:90, at_version=61'54, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.922378352999999, 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018170465, 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.775 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816924+0000", 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.816986+0000", 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1818999999999999e-05 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817088+0000", 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102377 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817111+0000", 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2946e-05 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817267+0000", 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015629899999999999 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.817294+0000", 2024-09-16T06:49:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6412999999999999e-05 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835044+0000", 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017750359 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835095+0000", 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1004999999999997e-05 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/37 ECSubWrite(tid=91, reqid=osd.1.0:91, at_version=61'56, trim_to=0'0, roll_forward_to=57'52))", 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.919334411999998, 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015175777999999999, 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.780 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819968+0000", 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.819967+0000", 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820019+0000", 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2649999999999999e-05 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820184+0000", 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016481499999999999 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820207+0000", 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.245e-05 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820328+0000", 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120942 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.820352+0000", 2024-09-16T06:49:38.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4732999999999999e-05 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835111+0000", 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014758378000000001 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:48.835144+0000", 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3547000000000003e-05 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/37 ECSubWrite(tid=189, reqid=osd.11.0:189, at_version=71'80, trim_to=0'0, roll_forward_to=69'78))", 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.907365476000003, 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016104653999999999, 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831937+0000", 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.831994+0000", 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7025999999999999e-05 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832116+0000", 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012219299999999999 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832140+0000", 2024-09-16T06:49:38.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4023e-05 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832493+0000", 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035320700000000002 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.832518+0000", 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5147000000000002e-05 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847982+0000", 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015463289 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848042+0000", 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0463e-05 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.789 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.es4 71/37 ECSubWrite(tid=22, reqid=osd.8.0:22, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.904871604999997, 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019478348999999999, 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834431+0000", 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834430+0000", 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834474+0000", 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3739999999999998e-05 2024-09-16T06:49:38.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834597+0000", 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122753 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834618+0000", 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1313000000000001e-05 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834923+0000", 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030448399999999999 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.834948+0000", 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5293e-05 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853837+0000", 2024-09-16T06:49:38.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018889153999999998 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853910+0000", 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2368999999999998e-05 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=144, reqid=client.4511.0:542, at_version=71'6, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.898357173000001, 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016044704, 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840946+0000", 2024-09-16T06:49:38.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.840944+0000", 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841000+0000", 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5287000000000001e-05 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841092+0000", 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2009000000000006e-05 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841121+0000", 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.885e-05 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841388+0000", 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026696100000000002 2024-09-16T06:49:38.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841425+0000", 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7626999999999997e-05 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856931+0000", 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015505755 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.856990+0000", 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9401999999999998e-05 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.897779079999999, 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016485414, 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:38.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841524+0000", 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841522+0000", 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841612+0000", 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0575999999999995e-05 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841764+0000", 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015128200000000001 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841786+0000", 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1759999999999998e-05 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848074+0000", 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062882529999999997 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.848099+0000", 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5510000000000001e-05 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857929+0000", 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098292880000000003 2024-09-16T06:49:38.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858009+0000", 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0599000000000005e-05 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=146, reqid=client.4511.0:543, at_version=71'7, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.897011413000001, 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018827678, 2024-09-16T06:49:38.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842291+0000", 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842289+0000", 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842343+0000", 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842426+0000", 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3009000000000004e-05 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.842448+0000", 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1860000000000001e-05 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850355+0000", 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079069680000000003 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.850377+0000", 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2167e-05 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861052+0000", 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010674678 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861119+0000", 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7303000000000003e-05 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.6s3 71/37 ECSubWrite(tid=147, reqid=client.4511.0:544, at_version=71'8, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:38.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.893375155999998, 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015229647000000001, 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845928+0000", 2024-09-16T06:49:38.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845926+0000", 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.845972+0000", 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5745999999999999e-05 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846062+0000", 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9909999999999998e-05 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.846133+0000", 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1109e-05 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853380+0000", 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072475899999999999 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.853426+0000", 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5456e-05 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861126+0000", 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077004730000000002 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.861157+0000", 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1130000000000002e-05 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=132, reqid=osd.1.0:132, at_version=72'83, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.895968021999998, 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015387283999999999, 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843335+0000", 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.843334+0000", 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:38.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844171+0000", 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083714800000000002 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845132+0000", 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096062600000000001 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845482+0000", 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035013699999999999 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845634+0000", 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015229600000000001 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845661+0000", 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:38.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858652+0000", 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012991745000000001 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.858722+0000", 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9822999999999999e-05 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.1s6 72/37 ECSubWrite(tid=133, reqid=osd.1.0:133, at_version=72'36, trim_to=0'0, roll_forward_to=71'34))", 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.895118668000002, 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017512679999999999, 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844184+0000", 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844183+0000", 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844237+0000", 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4425999999999997e-05 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845784+0000", 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015470320000000001 2024-09-16T06:49:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845809+0000", 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.455e-05 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845934+0000", 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012515900000000001 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.845958+0000", 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3879999999999998e-05 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861640+0000", 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01568172 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861697+0000", 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7173000000000003e-05 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s3 72/37 ECSubWrite(tid=134, reqid=osd.1.0:134, at_version=72'85, trim_to=0'0, roll_forward_to=71'81))", 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.894383869999999, 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017033627999999999, 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844919+0000", 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844918+0000", 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.844980+0000", 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1920000000000003e-05 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846382+0000", 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014028630000000001 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.846403+0000", 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0143e-05 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848953+0000", 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002550667 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.848977+0000", 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3337e-05 2024-09-16T06:49:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861911+0000", 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012934783 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.861953+0000", 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1192000000000001e-05 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs4 106/105 ECSubWrite(tid=336, reqid=client.4511.0:934, at_version=0'0, trim_to=0'0, roll_forward_to=106'164))", 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.869584+0000", 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.8697185760000004, 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049350335000000002, 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.869584+0000", 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.869584+0000", 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.869582+0000", 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.869647+0000", 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4282000000000006e-05 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.917821+0000", 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048174650999999999 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.917846+0000", 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.476e-05 2024-09-16T06:49:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.917948+0000", 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101574 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.917963+0000", 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5288e-05 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.918863+0000", 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090006000000000005 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.918935+0000", 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1539999999999996e-05 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:38.831 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:38.833 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T06:49:38.957 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:38.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:38.957 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:38.958 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:38.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T06:49:39.056 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:49:39.099 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:39.100 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:39.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.584016444, 2024-09-16T06:49:39.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76549419299999999, 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:39.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650577+0000", 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650574+0000", 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:39.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650608+0000", 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3713000000000001e-05 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650642+0000", 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4193999999999998e-05 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650679+0000", 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7076000000000002e-05 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650810+0000", 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130975 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:07.650839+0000", 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247e-05 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413038+0000", 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.76219903099999997 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413484+0000", 2024-09-16T06:49:39.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000445697 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.413899+0000", 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041460999999999999 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415002+0000", 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011036749999999999 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.415927+0000", 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092502600000000002 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416016+0000", 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8597000000000006e-05 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:08.416071+0000", 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5121999999999998e-05 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.770216266000006, 2024-09-16T06:49:39.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019417113, 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464377+0000", 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464375+0000", 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464390+0000", 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4669e-05 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464409+0000", 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8715e-05 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464426+0000", 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6762999999999999e-05 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.472876+0000", 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084499889999999998 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473092+0000", 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215971 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.473458+0000", 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036655500000000002 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.477925+0000", 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044670730000000002 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483564+0000", 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056391460000000003 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483680+0000", 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115585 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483794+0000", 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114343 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.770087713999999, 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019570884, 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-16T06:49:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464506+0000", 2024-09-16T06:49:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464505+0000", 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464513+0000", 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7100000000000007e-06 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464524+0000", 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1841999999999999e-05 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464534+0000", 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5259999999999999e-06 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474692+0000", 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010157668 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.474912+0000", 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022038200000000001 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.475242+0000", 2024-09-16T06:49:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033026699999999997 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480387+0000", 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051445429999999997 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483913+0000", 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035258849999999999 2024-09-16T06:49:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484016+0000", 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102942 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484077+0000", 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1016e-05 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.770050386999998, 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019858199, 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464543+0000", 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464542+0000", 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464643+0000", 2024-09-16T06:49:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101393 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464656+0000", 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2605999999999999e-05 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464665+0000", 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5860000000000005e-06 2024-09-16T06:49:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476022+0000", 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011357009 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476200+0000", 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178684 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.476393+0000", 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019251699999999999 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482510+0000", 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061175470000000001 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484189+0000", 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016784910000000001 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484303+0000", 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114479 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484401+0000", 2024-09-16T06:49:39.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7880000000000005e-05 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.769915389000005, 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019947972000000001, 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464678+0000", 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464677+0000", 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464686+0000", 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7390000000000005e-06 2024-09-16T06:49:39.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464695+0000", 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464702+0000", 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8600000000000004e-06 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478022+0000", 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013319589999999999 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478158+0000", 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136198 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.478416+0000", 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025765500000000003 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482532+0000", 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004115891 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484516+0000", 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019846870000000002 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484579+0000", 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2409000000000005e-05 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484626+0000", 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7296999999999998e-05 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.769883227999998, 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020226436, 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464710+0000", 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464715+0000", 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-06 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464724+0000", 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3230000000000004e-06 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464731+0000", 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479024+0000", 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014292787 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479239+0000", 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021545200000000001 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.479577+0000", 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033779699999999998 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.483830+0000", 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042531009999999996 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484719+0000", 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088904299999999995 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484840+0000", 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121221 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484937+0000", 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6206999999999996e-05 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.769854796999994, 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020646820999999999, 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464739+0000", 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464738+0000", 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464817+0000", 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9078000000000004e-05 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464828+0000", 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0532e-05 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464837+0000", 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0070000000000003e-06 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480460+0000", 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015622959 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.480685+0000", 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022519400000000001 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.481074+0000", 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038934700000000001 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484433+0000", 2024-09-16T06:49:39.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033589129999999998 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485277+0000", 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084439499999999995 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485339+0000", 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1356e-05 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.485385+0000", 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.672e-05 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.769747878000004, 2024-09-16T06:49:39.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29906042300000002, 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464846+0000", 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464845+0000", 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464851+0000", 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4659999999999996e-06 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464861+0000", 2024-09-16T06:49:39.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5689999999999994e-06 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.464868+0000", 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9369999999999998e-06 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482631+0000", 2024-09-16T06:49:39.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017763431999999999 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482762+0000", 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013073100000000001 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.482951+0000", 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000189567 2024-09-16T06:49:39.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.484968+0000", 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020167509999999998 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763764+0000", 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.27879603600000002 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763849+0000", 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4963999999999995e-05 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:09.763906+0000", 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6755999999999997e-05 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:186 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.076241366999994, 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021508751999999999, 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158352+0000", 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158351+0000", 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158358+0000", 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6270000000000002e-06 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158367+0000", 2024-09-16T06:49:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3919999999999992e-06 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158375+0000", 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171417+0000", 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013042410000000001 2024-09-16T06:49:39.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171642+0000", 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022488799999999999 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.171990+0000", 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000348303 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179615+0000", 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076250019999999996 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179642+0000", 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7328e-05 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179766+0000", 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123588 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.179861+0000", 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.076211119000007, 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022008608999999998, 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:39.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158382+0000", 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158388+0000", 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-16T06:49:39.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158397+0000", 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158404+0000", 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1400000000000002e-06 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173548+0000", 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015143536000000001 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173688+0000", 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140674 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.173885+0000", 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019667999999999999 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180001+0000", 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061158690000000003 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180128+0000", 2024-09-16T06:49:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012743800000000001 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180239+0000", 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111159 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180391+0000", 2024-09-16T06:49:39.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015157699999999999 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.076181684000005, 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023233196000000001, 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 188 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158412+0000", 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158411+0000", 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158496+0000", 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4388000000000006e-05 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158506+0000", 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713000000000001e-05 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158514+0000", 2024-09-16T06:49:39.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8159999999999994e-06 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.174815+0000", 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016301182000000001 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175095+0000", 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027987800000000001 2024-09-16T06:49:39.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.175447+0000", 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035187400000000001 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180109+0000", 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004661999 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181392+0000", 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283347 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181541+0000", 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014824 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.181645+0000", 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010439599999999999 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4185.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.076070764999997, 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700359000000002, 2024-09-16T06:49:39.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4185", 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/899211195", 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158523+0000", 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158522+0000", 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158528+0000", 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158537+0000", 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2499999999999995e-06 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.158545+0000", 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2690000000000002e-06 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180471+0000", 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021926737000000002 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180599+0000", 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012754800000000001 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.180795+0000", 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019607700000000001 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.182072+0000", 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012771320000000001 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183008+0000", 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093566400000000001 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183131+0000", 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000123225 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:10.183223+0000", 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1904000000000006e-05 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:77 3.8s0 3:119b2380:::smithi11731613-26:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.262901239000001, 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055956466000000003, 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.324 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 77 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:39.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971692+0000", 2024-09-16T06:49:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971691+0000", 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971701+0000", 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8519999999999999e-06 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971715+0000", 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4386000000000001e-05 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971725+0000", 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0360000000000001e-05 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.974948+0000", 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003222759 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975027+0000", 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8497000000000004e-05 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975520+0000", 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049320500000000003 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.977378+0000", 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018576230000000001 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027491+0000", 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050112973999999998 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027649+0000", 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015816800000000001 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:78 3.8s0 3:119b2380:::smithi11731613-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.262806225000006, 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059293289999999998, 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971787+0000", 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971786+0000", 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971793+0000", 2024-09-16T06:49:39.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971809+0000", 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5225000000000001e-05 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.971820+0000", 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0922e-05 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.975891+0000", 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040709759999999996 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.976067+0000", 2024-09-16T06:49:39.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017645900000000001 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027711+0000", 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051643840000000003 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.027855+0000", 2024-09-16T06:49:39.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014441399999999999 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.031081+0000", 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032252359999999998 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 35/32 ECSubWriteReply(tid=162, last_complete=35'4, committed=1, applied=1))", 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.201840762000003, 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48350030599999999, 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032753+0000", 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032752+0000", 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.032763+0000", 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1807000000000001e-05 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516055+0000", 2024-09-16T06:49:39.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.48329140799999998 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516068+0000", 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2782999999999999e-05 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516198+0000", 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129949 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.516253+0000", 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5390000000000003e-05 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 35/32 ECSubWrite(tid=17, reqid=client.4511.0:134, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.178744498, 2024-09-16T06:49:39.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.51123769299999999, 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055849+0000", 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055848+0000", 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.055863+0000", 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.464e-05 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565781+0000", 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50991880999999994 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565796+0000", 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4263e-05 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565932+0000", 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013609900000000001 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.565949+0000", 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7229000000000001e-05 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567019+0000", 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001070463 2024-09-16T06:49:39.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.567087+0000", 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7229000000000003e-05 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.7s2 40/37 ECSubRead(tid=11, to_read={3:e60b6e1b:::smithi11731613-14:head=0,4096,0}, subchunks={3:e60b6e1b:::smithi11731613-14:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:39.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 71.167063103000004, 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.375282063, 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067530+0000", 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067529+0000", 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.067547+0000", 2024-09-16T06:49:39.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8187000000000001e-05 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442125+0000", 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.37457846099999997 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442142+0000", 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6843000000000001e-05 2024-09-16T06:49:39.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442313+0000", 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017038200000000001 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:28.442812+0000", 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049983999999999998 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.4s4 71/37 ECSubWrite(tid=126, reqid=osd.1.0:126, at_version=71'77, trim_to=0'0, roll_forward_to=69'73))", 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.393029857999998, 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030302045, 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841564+0000", 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841562+0000", 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841613+0000", 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1739e-05 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841692+0000", 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8974000000000007e-05 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.841718+0000", 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5452999999999998e-05 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847063+0000", 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053447629999999998 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847087+0000", 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4553999999999999e-05 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871791+0000", 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024703433 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871866+0000", 2024-09-16T06:49:39.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5133e-05 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.0s4 71/32 ECSubWrite(tid=63, reqid=osd.12.0:63, at_version=71'48, trim_to=0'0, roll_forward_to=64'44))", 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.391365978000003, 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028623895, 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843227+0000", 2024-09-16T06:49:39.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843226+0000", 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843302+0000", 2024-09-16T06:49:39.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5922999999999998e-05 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843397+0000", 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5536000000000001e-05 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.843419+0000", 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2240000000000001e-05 2024-09-16T06:49:39.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847145+0000", 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037257950000000001 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.847167+0000", 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1590000000000002e-05 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871800+0000", 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024633057 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.871851+0000", 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1647000000000002e-05 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:564 3.8s0 3:1cef29f0:::smithi11731613-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.259110974000002, 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020010844, 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 564 2024-09-16T06:49:39.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975482+0000", 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975481+0000", 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975500+0000", 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975518+0000", 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332999999999999e-05 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.975531+0000", 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3207000000000001e-05 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.983862+0000", 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083305979999999998 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.984001+0000", 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000139629 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990097+0000", 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060955419999999998 2024-09-16T06:49:39.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.990219+0000", 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122349 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.995493+0000", 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052739930000000003 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.370 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:39.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T06:49:39.501 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:39.501 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:39.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T06:49:39.637 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:39.638 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:39.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:24 3.3s0 3:ce1274e9:::smithi11731613-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.876114900000005, 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033126111999999999, 2024-09-16T06:49:39.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901164+0000", 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901162+0000", 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901169+0000", 2024-09-16T06:49:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901187+0000", 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8320000000000001e-05 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901203+0000", 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5872999999999999e-05 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907294+0000", 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006090949 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907512+0000", 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021778499999999999 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924007+0000", 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016495668000000002 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924218+0000", 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021115199999999999 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934290+0000", 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010071302000000001 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:31 3.3s0 3:c7654c4c:::smithi11731613-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.876062576999999, 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026931531000000002, 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901216+0000", 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.901215+0000", 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902241+0000", 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010258190000000001 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902275+0000", 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4037e-05 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902312+0000", 2024-09-16T06:49:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.718e-05 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.907678+0000", 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053663829999999997 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.908030+0000", 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035200800000000001 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912004+0000", 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039736770000000001 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.923909+0000", 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011905429 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.927953+0000", 2024-09-16T06:49:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040441499999999998 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928147+0000", 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000194005 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:32 3.3s0 3:c7654c4c:::smithi11731613-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.874946519000005, 2024-09-16T06:49:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028234555000000001, 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 32 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902332+0000", 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902330+0000", 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902341+0000", 2024-09-16T06:49:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1279e-05 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902360+0000", 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239999999999999e-05 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902374+0000", 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3917000000000001e-05 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917230+0000", 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014855607 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917486+0000", 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025648799999999997 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.918644+0000", 2024-09-16T06:49:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011572749999999999 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.928363+0000", 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097195389999999993 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930283+0000", 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919505 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.930566+0000", 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028369099999999998 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:33 3.3s0 3:c7654c4c:::smithi11731613-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.874891829000006, 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037765197, 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 33 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902387+0000", 2024-09-16T06:49:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902386+0000", 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902392+0000", 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902410+0000", 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7893999999999998e-05 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902429+0000", 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8689999999999999e-05 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934639+0000", 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032210404999999998 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.934843+0000", 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020376199999999999 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.940152+0000", 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053090200000000002 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.2s5 35/32 ECSubWrite(tid=4, reqid=client.4511.0:44, at_version=35'4, trim_to=0'0, roll_forward_to=35'1))", 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.859975143, 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788766000000001, 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917303+0000", 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917302+0000", 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.917349+0000", 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7796000000000001e-05 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941504+0000", 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024154637 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941525+0000", 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0608999999999999e-05 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941640+0000", 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011531000000000001 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.941662+0000", 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2328999999999999e-05 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943041+0000", 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001379047 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943092+0000", 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0834000000000002e-05 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:68 3.3s0 3:cbbd057c:::smithi11731613-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.823253528999999, 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045964098000000002, 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 68 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954025+0000", 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954024+0000", 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954031+0000", 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954046+0000", 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4606999999999999e-05 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954058+0000", 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1938e-05 2024-09-16T06:49:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956544+0000", 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024857949999999998 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.956742+0000", 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019819 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.957650+0000", 2024-09-16T06:49:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090843 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959725+0000", 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074256 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999914+0000", 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040189895000000003 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999989+0000", 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4527e-05 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:69 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.823209043999995, 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048288992000000003, 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 69 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954069+0000", 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954068+0000", 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954074+0000", 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954088+0000", 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4866e-05 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.954100+0000", 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1328999999999999e-05 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959103+0000", 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035749999999997 2024-09-16T06:49:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.959283+0000", 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017953899999999999 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000081+0000", 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040797893000000002 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.000261+0000", 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180304 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002358+0000", 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002097392 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:98 3.3s0 3:c742dae1:::smithi11731613-33:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.781785178000007, 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.086611296000000004, 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 98 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995493+0000", 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995492+0000", 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995501+0000", 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8720000000000008e-06 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995517+0000", 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6650000000000002e-05 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995529+0000", 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1447999999999999e-05 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.997874+0000", 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023456250000000001 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998037+0000", 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016212600000000001 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.998796+0000", 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075897099999999999 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.002377+0000", 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00358195 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081972+0000", 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.079594867 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082105+0000", 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013220899999999999 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:99 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.781718115000004, 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.087893389000000002, 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 99 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995560+0000", 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995559+0000", 2024-09-16T06:49:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995565+0000", 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2659999999999998e-06 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995581+0000", 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.611e-05 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.995594+0000", 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2269000000000001e-05 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999116+0000", 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003522238 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.999227+0000", 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111483 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082346+0000", 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.083118743999999994 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082416+0000", 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9993000000000006e-05 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.083454+0000", 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00103751 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:158 3.3s0 3:c7654c4c:::smithi11731613-11:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.697492304999997, 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41982487699999999, 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079786+0000", 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079784+0000", 2024-09-16T06:49:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079794+0000", 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6260000000000007e-06 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079810+0000", 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6631000000000001e-05 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079821+0000", 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0849000000000001e-05 2024-09-16T06:49:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.079917+0000", 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5730999999999995e-05 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.080126+0000", 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020895399999999999 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.081069+0000", 2024-09-16T06:49:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094364700000000004 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.082604+0000", 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001534863 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499525+0000", 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.41692029400000002 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:23.499611+0000", 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-05 2024-09-16T06:49:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:226 3.3s0 3:ce1274e9:::smithi11731613-8:head [append 0~0] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 71.821611504000003, 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027980046000000001, 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 226 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955667+0000", 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955665+0000", 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955672+0000", 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000002e-06 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955690+0000", 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8179999999999999e-05 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955704+0000", 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.396e-05 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956020+0000", 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000316101 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956038+0000", 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8425999999999999e-05 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974108+0000", 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018069776999999999 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.974338+0000", 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022928799999999999 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.975586+0000", 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012484639999999999 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.979590+0000", 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040043939999999997 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983481+0000", 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038905480000000002 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.983647+0000", 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000166045 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:351 3.3s0 3:c742dae1:::smithi11731613-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.935325124000002, 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023956603, 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 351 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841953+0000", 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841957+0000", 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0199999999999996e-06 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841966+0000", 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-06 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.841974+0000", 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.34e-06 2024-09-16T06:49:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.849799+0000", 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078258089999999995 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.850139+0000", 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033906799999999999 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862313+0000", 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012174318999999999 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.862630+0000", 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031676099999999998 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.865910+0000", 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032802859999999999 2024-09-16T06:49:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.as3 47/32 ECSubWrite(tid=68, reqid=client.4511.0:361, at_version=0'0, trim_to=0'0, roll_forward_to=47'47))", 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.905761032000001, 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.50083384399999997, 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871517+0000", 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871516+0000", 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:34.871541+0000", 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5782000000000002e-05 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370610+0000", 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.49906899100000002 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370625+0000", 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.429e-05 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370744+0000", 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119746 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.370773+0000", 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8807e-05 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372302+0000", 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015286519999999999 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:35.372351+0000", 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9348999999999998e-05 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 73/32 ECSubWriteReply(tid=108, last_complete=73'90, committed=1, applied=1))", 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.907518510000003, 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071784013999999993, 2024-09-16T06:49:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869760+0000", 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869759+0000", 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.869765+0000", 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7590000000000003e-06 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941447+0000", 2024-09-16T06:49:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071682318999999994 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941458+0000", 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1099e-05 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941509+0000", 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1201e-05 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.941544+0000", 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4538e-05 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:971 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~5586944,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:32.868850+0000", 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.9084280710000003, 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027643053000000001, 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 971 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868850+0000", 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868850+0000", 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868847+0000", 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868867+0000", 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.999e-05 2024-09-16T06:49:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868901+0000", 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4819e-05 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868922+0000", 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0775999999999999e-05 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869063+0000", 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014036300000000001 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869352+0000", 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000289611 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.896493+0000", 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027141209999999999 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:972 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~5586944,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:32.868998+0000", 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.9082800149999999, 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033362817000000003, 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 972 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868998+0000", 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868998+0000", 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868997+0000", 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869026+0000", 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8575999999999999e-05 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869049+0000", 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3272999999999999e-05 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869066+0000", 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.734e-05 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.870215+0000", 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011492220000000001 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.870389+0000", 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017363099999999999 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.902361+0000", 2024-09-16T06:49:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031972238 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:973 3.3s0 3:cbbd057c:::smithi11731613-23:head [read 0~5586944,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:32.869083+0000", 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.9081957660000004, 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037295206999999997, 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 973 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869083+0000", 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869083+0000", 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869081+0000", 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869090+0000", 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0170000000000002e-06 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869108+0000", 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8423e-05 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869122+0000", 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3933e-05 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.880749+0000", 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011626427 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.881126+0000", 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037705699999999998 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.906378+0000", 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025252159999999999 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s5 105/77 ECSubWrite(tid=239, reqid=client.4511.0:978, at_version=105'166, trim_to=0'0, roll_forward_to=105'163))", 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:32.911325+0000", 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.8659531310000004, 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.65136649700000004, 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.911325+0000", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.911325+0000", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.911324+0000", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.911364+0000", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9854000000000002e-05 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.561471+0000", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.65010778499999999 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.561494+0000", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2637e-05 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.561605+0000", 2024-09-16T06:49:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110769 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.561626+0000", 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1333000000000001e-05 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.562633+0000", 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010071769999999999 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.562692+0000", 2024-09-16T06:49:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8505000000000003e-05 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.2s5 106/105 ECSubRead(tid=210, to_read={3:4eee07bc:::smithi11731613-24:head=0,262144,0}, subchunks={3:4eee07bc:::smithi11731613-24:head=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.907620+0000", 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.8696586279999998, 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047531878999999999, 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.907620+0000", 2024-09-16T06:49:39.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.907620+0000", 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.907618+0000", 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.907633+0000", 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4182000000000001e-05 2024-09-16T06:49:39.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953442+0000", 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045809227000000001 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953461+0000", 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9493000000000001e-05 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953649+0000", 2024-09-16T06:49:39.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018789300000000001 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.955152+0000", 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001502499 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpRead(3.2s5 106/105 ECSubRead(tid=217, to_read={3:476f74d7:::smithi11731613-27:3c=0,262144,0}, subchunks={3:476f74d7:::smithi11731613-27:3c=[0,1]}, attrs_to_read=))", 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.953338+0000", 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.8239400630000002, 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.54600097400000003, 2024-09-16T06:49:39.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953338+0000", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953338+0000", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953336+0000", 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:39.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.953361+0000", 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5316e-05 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:34.498001+0000", 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.544639867 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:34.498019+0000", 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8493999999999999e-05 2024-09-16T06:49:39.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:34.498189+0000", 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016985200000000001 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:34.499339+0000", 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011500449999999999 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:39.873 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:39.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-16T06:49:40.005 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:40.005 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:40.005 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-16T06:49:40.005 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:40.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-16T06:49:40.135 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:40.136 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-16T06:49:40.136 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-16T06:49:40.136 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-16T06:49:40.136 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:40.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-16T06:49:40.275 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-16T06:49:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-16T06:49:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:5 3.2s0 3:42d6f0b3:::smithi11731613-2:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.380884777999995, 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029483957000000002, 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-16T06:49:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892375+0000", 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892373+0000", 2024-09-16T06:49:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892385+0000", 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1939999999999999e-05 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892403+0000", 2024-09-16T06:49:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8202999999999998e-05 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892421+0000", 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8189000000000002e-05 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895208+0000", 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786527 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.895392+0000", 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000184255 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.896349+0000", 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00095644999999999997 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.902998+0000", 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066495039999999997 2024-09-16T06:49:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921651+0000", 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018652716 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.921859+0000", 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208359 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:6 3.2s0 3:42d6f0b3:::smithi11731613-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.380803599000004, 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034217268000000002, 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6 2024-09-16T06:49:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892456+0000", 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892455+0000", 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.999999 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892464+0000", 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.67e-06 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892484+0000", 2024-09-16T06:49:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0265999999999999e-05 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.892500+0000", 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.596e-05 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897287+0000", 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047868049999999999 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.897527+0000", 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023998800000000001 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922181+0000", 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024653615 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.922389+0000", 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020808600000000001 2024-09-16T06:49:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926674+0000", 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042850639999999999 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:43 3.2s0 3:4636348c:::smithi11731613-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.367585546000001, 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020346564000000001, 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905674+0000", 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.905671+0000", 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909244+0000", 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035731700000000001 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909296+0000", 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2352999999999998e-05 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909321+0000", 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4813000000000001e-05 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909751+0000", 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042994099999999998 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.910155+0000", 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040433999999999999 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.912510+0000", 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002354929 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924742+0000", 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012231967 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.925839+0000", 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001096302 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.926021+0000", 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000182492 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:44 3.2s0 3:4636348c:::smithi11731613-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.363899915999994, 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034540078000000002, 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909360+0000", 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909357+0000", 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909370+0000", 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2583e-05 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909390+0000", 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9930000000000001e-05 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909421+0000", 2024-09-16T06:49:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1285999999999998e-05 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913550+0000", 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041288399999999999 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.913768+0000", 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000217691 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.914791+0000", 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010230770000000001 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.924746+0000", 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099553599999999999 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943736+0000", 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018990000999999999 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.943900+0000", 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016405 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:45 3.2s0 3:4636348c:::smithi11731613-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.363824980000004, 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040625472000000003, 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-16T06:49:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909435+0000", 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909434+0000", 2024-09-16T06:49:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909441+0000", 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0330000000000003e-06 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909460+0000", 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8674000000000001e-05 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.909473+0000", 2024-09-16T06:49:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3485999999999999e-05 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919590+0000", 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010117482000000001 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.919812+0000", 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000221389 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944680+0000", 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024868369000000001 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.944880+0000", 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019949900000000001 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:22.950061+0000", 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051807770000000001 2024-09-16T06:49:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:231 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.317537369999997, 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021272859000000002, 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 231 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955723+0000", 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955721+0000", 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955733+0000", 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2129e-05 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955755+0000", 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1423000000000001e-05 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955770+0000", 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4790000000000001e-05 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955974+0000", 2024-09-16T06:49:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020403199999999999 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955982+0000", 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6940000000000001e-06 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.967793+0000", 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011811056 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.968023+0000", 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023004500000000001 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.976996+0000", 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089721929999999998 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:232 3.2s0 3:4636348c:::smithi11731613-15:2 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.317312825000002, 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022137836000000001, 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 232 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955947+0000", 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955946+0000", 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955958+0000", 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-05 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955977+0000", 2024-09-16T06:49:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.912e-05 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.955993+0000", 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5894000000000001e-05 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956101+0000", 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107402 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.956123+0000", 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2056999999999999e-05 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971700+0000", 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015577155000000001 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.971931+0000", 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023091400000000001 2024-09-16T06:49:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:27.978085+0000", 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061543179999999998 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 55/37 ECSubWrite(tid=121, reqid=client.4511.0:431, at_version=55'56, trim_to=0'0, roll_forward_to=52'54))", 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.455697731999997, 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.102329752, 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817562+0000", 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.817619+0000", 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7778999999999997e-05 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917448+0000", 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.099828728000000005 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917466+0000", 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7907e-05 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917613+0000", 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014708200000000001 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.917643+0000", 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0315999999999999e-05 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919830+0000", 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021864940000000002 2024-09-16T06:49:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:42.919892+0000", 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2076000000000007e-05 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.fs3 71/37 ECSubWrite(tid=192, reqid=osd.11.0:192, at_version=71'84, trim_to=0'0, roll_forward_to=65'80))", 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.436581412999999, 2024-09-16T06:49:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022295459, 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836679+0000", 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836677+0000", 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836728+0000", 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1388999999999999e-05 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836823+0000", 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4826000000000003e-05 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836845+0000", 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2310000000000002e-05 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836965+0000", 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119603 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.836989+0000", 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4258999999999999e-05 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.857456+0000", 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020466868999999999 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:58.858974+0000", 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015182590000000001 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=71, reqid=client.4511.0:549, at_version=72'66, trim_to=0'0, roll_forward_to=72'64))", 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.375255903999999, 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051205101000000003, 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898004+0000", 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898003+0000", 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.898034+0000", 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1466e-05 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948063+0000", 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050028524999999997 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948079+0000", 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5869999999999999e-05 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948191+0000", 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112663 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.948206+0000", 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.484e-05 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.949167+0000", 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096042199999999995 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.949209+0000", 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2706000000000001e-05 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=77, reqid=client.4511.0:555, at_version=72'72, trim_to=0'0, roll_forward_to=72'70))", 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.307155604000002, 2024-09-16T06:49:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036435799999999997, 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966104+0000", 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966102+0000", 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:48:59.966153+0000", 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0362999999999999e-05 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000163+0000", 2024-09-16T06:49:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034010037999999999 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000187+0000", 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4159e-05 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000326+0000", 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013869500000000001 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.000356+0000", 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0511999999999999e-05 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.002480+0000", 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021234090000000001 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.002540+0000", 2024-09-16T06:49:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0733999999999998e-05 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWrite(3.7s4 72/37 ECSubWrite(tid=82, reqid=client.4511.0:572, at_version=72'77, trim_to=0'0, roll_forward_to=72'72))", 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.261032651000001, 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.301395462, 2024-09-16T06:49:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012227+0000", 2024-09-16T06:49:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012226+0000", 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.012275+0000", 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8619999999999999e-05 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.302391+0000", 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.29011663999999998 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.302400+0000", 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9409999999999996e-06 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.305565+0000", 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031649389999999999 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.305627+0000", 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1673000000000004e-05 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.313590+0000", 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079632660000000001 2024-09-16T06:49:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.313623+0000", 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2855000000000003e-05 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 72/32 ECSubWriteReply(tid=114, last_complete=72'106, committed=1, applied=1))", 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.948611374999999, 2024-09-16T06:49:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.17086979499999999, 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324649+0000", 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324647+0000", 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.324660+0000", 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2982999999999999e-05 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495344+0000", 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.17068462700000001 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495360+0000", 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5539000000000001e-05 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495453+0000", 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3367000000000002e-05 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:00.495518+0000", 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5278999999999996e-05 2024-09-16T06:49:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:806 3.2s0 3:476f74d7:::smithi11731613-27:head [append 0~671744] snapc 35=[33,31,2e,2b,2a,28,21,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.442824979000001, 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022399347, 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 806 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830435+0000", 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.830432+0000", 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831511+0000", 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010790019999999999 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831552+0000", 2024-09-16T06:49:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1624e-05 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831570+0000", 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8056000000000001e-05 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831718+0000", 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014737600000000001 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.831744+0000", 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6622999999999999e-05 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836450+0000", 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047061350000000002 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.836721+0000", 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027029199999999999 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.838621+0000", 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019002820000000001 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.841994+0000", 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003373029 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.852763+0000", 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010769065 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:16.852834+0000", 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.127e-05 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:969 3.2s0 3:47b7cdf1:::smithi11731613-39:head [rollback 31] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:32.868294+0000", 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.4049663219999999, 2024-09-16T06:49:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.064651252, 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 969 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868294+0000", 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868294+0000", 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868291+0000", 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868307+0000", 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.596e-05 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868338+0000", 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1050000000000003e-05 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868359+0000", 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0970000000000001e-05 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869067+0000", 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070819800000000005 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869073+0000", 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.246e-06 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.876198+0000", 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0071247080000001 2024-09-16T06:49:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.876760+0000", 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056211900000000003 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for degraded object", 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.878246+0000", 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001485959 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.910404+0000", 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032158020000000002 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.910677+0000", 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027343000000000002 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.912140+0000", 2024-09-16T06:49:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014626979999999999 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.931670+0000", 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019530240000000001 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.932212+0000", 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054121300000000002 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.932945+0000", 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073335499999999999 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:970 3.2s0 3:47b7cdf1:::smithi11731613-39:head [append 0~0] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:32.868399+0000", 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.4048612900000004, 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.081797372, 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 970 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868399+0000", 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868399+0000", 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868398+0000", 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868406+0000", 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0360000000000006e-06 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868426+0000", 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0336999999999999e-05 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.868441+0000", 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5173e-05 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869160+0000", 2024-09-16T06:49:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00071850500000000003 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:32.869162+0000", 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1569999999999998e-06 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.891315+0000", 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0221532449999999 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for degraded object", 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.891642+0000", 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032666800000000001 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.943238+0000", 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051596858000000002 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.943883+0000", 2024-09-16T06:49:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00064420500000000002 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.945113+0000", 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012303609999999999 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.947506+0000", 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002392543 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.949504+0000", 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001998035 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.950196+0000", 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069248200000000002 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:990 3.2s0 3:42d6f0b3:::smithi11731613-2:head [append 0~0] snapc 47=[47,44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.839328+0000", 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4339325389999997, 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.061867907999999999, 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 990 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839328+0000", 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839328+0000", 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839325+0000", 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839340+0000", 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5583e-05 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839371+0000", 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0467000000000002e-05 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839392+0000", 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0820000000000001e-05 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.844767+0000", 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053756689999999996 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.844774+0000", 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.657e-06 2024-09-16T06:49:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.891961+0000", 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047186902000000003 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.892584+0000", 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062341499999999999 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.894718+0000", 2024-09-16T06:49:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021336609999999998 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.899575+0000", 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048568700000000001 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.900496+0000", 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092126600000000001 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.901195+0000", 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069943800000000003 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:991 3.2s0 3:42d6f0b3:::smithi11731613-2:head [rollback 18] snapc 47=[47,44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.839412+0000", 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4338481359999999, 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.107909332, 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 991 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839412+0000", 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839412+0000", 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839411+0000", 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839426+0000", 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5270000000000001e-05 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839445+0000", 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9026999999999999e-05 2024-09-16T06:49:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839459+0000", 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4506e-05 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.844857+0000", 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053981599999999999 2024-09-16T06:49:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.844862+0000", 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8300000000000003e-06 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.896671+0000", 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051809090000000002 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.896987+0000", 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031518799999999999 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for degraded object", 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.898756+0000", 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001769767 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.936647+0000", 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037890198999999999 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.936943+0000", 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029651500000000002 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.938580+0000", 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001636974 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.946147+0000", 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075668159999999996 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.946489+0000", 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034193 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.947321+0000", 2024-09-16T06:49:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083245400000000005 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4511.0:992 3.2s0 3:42d6f0b3:::smithi11731613-2:head [append 0~0] snapc 47=[47,44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.839473+0000", 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.4337868330000001, 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.109131906, 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4511", 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 992 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839473+0000", 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839473+0000", 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839472+0000", 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839479+0000", 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6100000000000002e-06 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839495+0000", 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6436e-05 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.839508+0000", 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3380000000000001e-05 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.844918+0000", 2024-09-16T06:49:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054099760000000004 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.844920+0000", 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.373e-06 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.899269+0000", 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054349355000000002 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for degraded object", 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.899542+0000", 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027303499999999998 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.940922+0000", 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041379590000000001 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.941229+0000", 2024-09-16T06:49:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030726100000000002 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.942551+0000", 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013217470000000001 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.947328+0000", 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004777646 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.947904+0000", 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057599599999999995 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.948605+0000", 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070071100000000002 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "MOSDPGPushReply(3.2s0 106/105 [PushReplyOp(3:42d6f0b3:::smithi11731613-2:19)])", 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-16T06:49:33.915421+0000", 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.357839469, 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021129918000000001, 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.915421+0000", 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.915421+0000", 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.915420+0000", 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.915426+0000", 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3239999999999999e-06 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.915555+0000", 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012874499999999999 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.915568+0000", 2024-09-16T06:49:40.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3329999999999999e-05 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.934174+0000", 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018605368000000001 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-16T06:49:33.936550+0000", 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023767549999999999 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-16T06:49:40.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-16T06:49:40.394 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-16T06:49:40.395 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T06:49:40.528 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:40.528 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:40.528 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:40.528 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:40.542 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T06:49:40.663 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:40.663 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:40.663 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:40.663 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:40.663 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:40.677 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:75 3.as0 3:5d679340:::smithi11731613-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.804 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.838639294000004, 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.033077179999999998, 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 75 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962305+0000", 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962304+0000", 2024-09-16T06:49:40.806 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962308+0000", 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0600000000000001e-06 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962314+0000", 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.088e-06 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.807 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.962319+0000", 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1120000000000004e-06 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.964978+0000", 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0026582540000000001 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.965065+0000", 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7108999999999996e-05 2024-09-16T06:49:40.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989270+0000", 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024205503 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.989359+0000", 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9204999999999997e-05 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995382+0000", 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0060223200000000003 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:95 3.as0 3:5bc02ad9:::smithi11731613-32:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.808754718000003, 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.042547695000000003, 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.810 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 95 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992189+0000", 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992188+0000", 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.812 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992194+0000", 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8760000000000002e-06 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992204+0000", 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.011e-05 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992212+0000", 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1890000000000001e-06 2024-09-16T06:49:40.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993832+0000", 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001620565 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.993886+0000", 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4021999999999999e-05 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994477+0000", 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059046499999999998 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995906+0000", 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014291919999999999 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034552+0000", 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.038646106999999999 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.815 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034737+0000", 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00018489900000000001 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:96 3.as0 3:5bc02ad9:::smithi11731613-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.808725734999996, 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.058410493000000001, 2024-09-16T06:49:40.816 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 96 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.817 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992218+0000", 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992221+0000", 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2899999999999998e-06 2024-09-16T06:49:40.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992229+0000", 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3080000000000008e-06 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.992236+0000", 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.528e-06 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.994995+0000", 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027588700000000001 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995064+0000", 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9758000000000005e-05 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035940+0000", 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040876066000000003 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.036021+0000", 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0243999999999999e-05 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.050629+0000", 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014608009999999999 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=18, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.805643986999996, 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.040520444000000003, 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.822 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995300+0000", 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995299+0000", 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.995304+0000", 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5680000000000001e-06 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034371+0000", 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.039067118999999997 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034380+0000", 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7949999999999996e-06 2024-09-16T06:49:40.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034448+0000", 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8244000000000006e-05 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035820+0000", 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001372306 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:144 3.as0 3:50fa89c4:::smithi11731613-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.737257518000007, 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38453364499999998, 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.826 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 144 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063686+0000", 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063691+0000", 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8879999999999996e-06 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.828 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063699+0000", 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5799999999999992e-06 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063705+0000", 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.246e-06 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065212+0000", 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015062820000000001 2024-09-16T06:49:40.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065312+0000", 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00010052600000000001 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.065706+0000", 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039373300000000001 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070934+0000", 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0052282070000000003 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448115+0000", 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37718074400000001 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448220+0000", 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105104 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.831 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:145 3.as0 3:50fa89c4:::smithi11731613-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.737214549000001, 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38866241499999998, 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.832 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 145 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063729+0000", 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063732+0000", 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1999999999999999e-06 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063741+0000", 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.4700000000000008e-06 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.063748+0000", 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4740000000000002e-06 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066083+0000", 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0023351140000000001 2024-09-16T06:49:40.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.066155+0000", 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1650999999999996e-05 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448700+0000", 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38254526700000002 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448854+0000", 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154306 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.452392+0000", 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0035374640000000001 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.837 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 35/32 ECSubWriteReply(tid=23, last_complete=35'13, committed=1, applied=1))", 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 77.729949339000001, 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37764362699999998, 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:40.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070995+0000", 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.070994+0000", 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.071000+0000", 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8980000000000001e-06 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447985+0000", 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.37698559799999998 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.447996+0000", 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0672999999999999e-05 2024-09-16T06:49:40.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448044+0000", 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8338999999999999e-05 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.448638+0000", 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059397499999999995 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:302 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 2] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.666081192000007, 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21732169600000001, 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 302 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134863+0000", 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134859+0000", 2024-09-16T06:49:40.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134878+0000", 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9264999999999999e-05 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134922+0000", 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3260000000000003e-05 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.844 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134944+0000", 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2603e-05 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135181+0000", 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00023672400000000001 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135385+0000", 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020434300000000001 2024-09-16T06:49:40.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.347993+0000", 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21260742799999999 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.351735+0000", 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037421830000000001 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352007+0000", 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027182200000000001 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.352184+0000", 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017777400000000001 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:303 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 70.665975021999998, 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21552337099999999, 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 303 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134969+0000", 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134967+0000", 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134976+0000", 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3860000000000007e-06 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.134996+0000", 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0234999999999999e-05 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135015+0000", 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8949000000000001e-05 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135785+0000", 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000770388 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:40.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.135925+0000", 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013959700000000001 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350066+0000", 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.21414113400000001 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350271+0000", 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020513000000000001 2024-09-16T06:49:40.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.350492+0000", 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022106200000000001 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.as0 61/32 ECSubWriteReply(tid=126, last_complete=61'99, committed=1, applied=1))", 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:40.853 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 51.925261384000002, 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63424799499999995, 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875682+0000", 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875681+0000", 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.875692+0000", 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1154e-05 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508662+0000", 2024-09-16T06:49:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.63297013099999999 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508675+0000", 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3217000000000001e-05 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.508759+0000", 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3652999999999993e-05 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:49.509930+0000", 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001171486 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:542 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.972532690999998, 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031510007, 2024-09-16T06:49:40.857 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 542 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.858 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828411+0000", 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828399+0000", 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999881 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828425+0000", 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5700000000000001e-05 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828469+0000", 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3510999999999997e-05 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828493+0000", 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4769999999999998e-05 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828815+0000", 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032114900000000001 2024-09-16T06:49:40.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828827+0000", 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2143999999999999e-05 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835549+0000", 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.006721856 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.835824+0000", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027592399999999998 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.838011+0000", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021865410000000002 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.849034+0000", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011023141 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859755+0000", 2024-09-16T06:49:40.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010721039999999999 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.859921+0000", 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016604799999999999 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:543 3.6s0 3:64026cd9:::smithi11731613-35:head [rollback 24] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.972414454999999, 2024-09-16T06:49:40.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034430898000000001, 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 543 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828529+0000", 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828528+0000", 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828538+0000", 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0457e-05 2024-09-16T06:49:40.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828557+0000", 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8459000000000001e-05 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828571+0000", 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.447e-05 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828929+0000", 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035788200000000001 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828944+0000", 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4662e-05 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840136+0000", 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011191942 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840414+0000", 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00027797100000000001 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.842038+0000", 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0016248160000000001 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855171+0000", 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013132853 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862552+0000", 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073803990000000002 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.862960+0000", 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00040862400000000001 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:544 3.6s0 3:64026cd9:::smithi11731613-35:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11,c] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.972359887000003, 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.034849585000000002, 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.870 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 544 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828584+0000", 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828583+0000", 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828589+0000", 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5819999999999998e-06 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828607+0000", 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7260000000000001e-05 2024-09-16T06:49:40.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828620+0000", 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3789e-05 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828991+0000", 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037102199999999999 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:40.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.828999+0000", 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5000000000000002e-06 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843716+0000", 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0147174 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843970+0000", 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000253732 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845224+0000", 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001253524 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.855177+0000", 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0099530999999999994 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863271+0000", 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0080939529999999992 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.863434+0000", 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00016288899999999999 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:743 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 34] snapc 2e=[2e,2b,2a,29,28,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 29.955767159000001, 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028808912999999998, 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.876 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 743 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845177+0000", 2024-09-16T06:49:40.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845175+0000", 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845184+0000", 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9590000000000001e-06 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845198+0000", 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4282e-05 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845208+0000", 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0258000000000001e-05 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845266+0000", 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7373000000000001e-05 2024-09-16T06:49:40.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.845364+0000", 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.8196000000000006e-05 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.869612+0000", 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.024247713000000001 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.871840+0000", 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022288270000000001 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.873933+0000", 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020920740000000002 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:10.873986+0000", 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3121000000000002e-05 2024-09-16T06:49:40.881 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.882 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.882 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s2 86/32 ECSubWrite(tid=148, reqid=client.4511.0:775, at_version=0'0, trim_to=0'0, roll_forward_to=86'141))", 2024-09-16T06:49:40.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 26.845448718, 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.83265717400000006, 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955495+0000", 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955494+0000", 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:40.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:13.955505+0000", 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0801e-05 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786476+0000", 2024-09-16T06:49:40.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.83097071499999997 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786491+0000", 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5109999999999999e-05 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786683+0000", 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000192165 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.786704+0000", 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0873000000000002e-05 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.788074+0000", 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001370079 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:14.788152+0000", 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8248e-05 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:852 3.as0 3:57ffea71:::smithi11731613-30:head [append 0~0] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.864444352, 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.029643135000000001, 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.889 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 852 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936500+0000", 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936496+0000", 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936509+0000", 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.205e-05 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936550+0000", 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1384000000000002e-05 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936564+0000", 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4053000000000001e-05 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936833+0000", 2024-09-16T06:49:40.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026950699999999999 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936856+0000", 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2518999999999999e-05 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.941811+0000", 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00495547 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.941931+0000", 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011981699999999999 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943080+0000", 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001149129 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.965859+0000", 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022778732999999999 2024-09-16T06:49:40.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966004+0000", 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014514199999999999 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966143+0000", 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000138391 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:853 3.as0 3:57ffea71:::smithi11731613-30:head [rollback 15] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.864358626000001, 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.029774785000000002, 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 853 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.896 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936585+0000", 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936584+0000", 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936591+0000", 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.365e-06 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936603+0000", 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2191999999999999e-05 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936612+0000", 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0350000000000007e-06 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936994+0000", 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038190700000000001 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937012+0000", 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8757e-05 2024-09-16T06:49:40.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.947015+0000", 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010002341999999999 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.947373+0000", 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00035835499999999998 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.950105+0000", 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027321899999999998 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966146+0000", 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016041161000000002 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966240+0000", 2024-09-16T06:49:40.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3199e-05 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966360+0000", 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012037 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:854 3.as0 3:57ffea71:::smithi11731613-30:head [append 0~0] snapc 3a=[3a,33,31,2e,28,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:40.902 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 18.864324399000001, 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.030019711000000001, 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 854 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936619+0000", 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936623+0000", 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1659999999999996e-06 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936650+0000", 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7022e-05 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936658+0000", 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.0809999999999994e-06 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937075+0000", 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000416628 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.937091+0000", 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6586e-05 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.951259+0000", 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014167170999999999 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.951679+0000", 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042059200000000001 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.952960+0000", 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012813690000000001 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966424+0000", 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013463298 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966523+0000", 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.9567000000000004e-05 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.966639+0000", 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000115837 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:961 3.as0 3:57ffea71:::smithi11731613-30:head [copy-from ver 94 in=55b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 7.9399251020000001, 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.092210185, 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.909 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 961 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861019+0000", 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861017+0000", 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861028+0000", 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1347e-05 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861059+0000", 2024-09-16T06:49:40.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0338999999999999e-05 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861078+0000", 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.98e-05 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.862868+0000", 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0017891809999999999 2024-09-16T06:49:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863085+0000", 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000217359 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.940308+0000", 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.077223240999999998 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:40.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.943429+0000", 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031204499999999999 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.952785+0000", 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0093566599999999993 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.953229+0000", 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00044375099999999999 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:962 3.as0 3:57ffea71:::smithi11731613-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 7.9398497030000001, 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.081297473999999995, 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 962 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.915 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861094+0000", 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861093+0000", 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861099+0000", 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6499999999999999e-06 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861122+0000", 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2629e-05 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.861136+0000", 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.437e-05 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863286+0000", 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021499359999999999 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.918 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.863365+0000", 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8294000000000006e-05 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.941742+0000", 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.078377418000000004 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.919 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.941953+0000", 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000210934 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.942392+0000", 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00043858299999999999 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:40.920 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:40.922 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T06:49:41.047 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:41.048 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:41.048 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:41.048 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:41.061 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T06:49:41.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T06:49:41.180 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:41.180 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:41.180 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:41.180 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:41.180 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:41.194 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:2 3.ds0 3:b273fc7d:::smithi11731613-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:41.319 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.423467450999993, 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023946957000000001, 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 2 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:41.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893110+0000", 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893109+0000", 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.321 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893127+0000", 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8227999999999999e-05 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893164+0000", 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6813999999999997e-05 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893180+0000", 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6178e-05 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.894904+0000", 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001723619 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895040+0000", 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013578799999999999 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.895729+0000", 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00068973600000000004 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.902324+0000", 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0065943900000000003 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.916978+0000", 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014654584 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917057+0000", 2024-09-16T06:49:41.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.8462999999999997e-05 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:3 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.423383654999995, 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027861434000000001, 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.325 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 3 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893194+0000", 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893192+0000", 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893200+0000", 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893222+0000", 2024-09-16T06:49:41.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1620999999999998e-05 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.893236+0000", 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4008999999999999e-05 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898109+0000", 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048738949999999996 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:41.328 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.898391+0000", 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000281716 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917365+0000", 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018973980000000001 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.917412+0000", 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6714e-05 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.329 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.921055+0000", 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003643112 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:50 3.ds0 3:b2c197ab:::smithi11731613-17:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.391978416000001, 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.028326324999999999, 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.330 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 50 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924599+0000", 2024-09-16T06:49:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924598+0000", 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924614+0000", 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5721000000000001e-05 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924626+0000", 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2456999999999999e-05 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924638+0000", 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2064e-05 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926064+0000", 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001426112 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926152+0000", 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7460000000000006e-05 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926580+0000", 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00042787800000000003 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927695+0000", 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00111517 2024-09-16T06:49:41.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952865+0000", 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.025170713000000001 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.952925+0000", 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9654999999999997e-05 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:51 3.ds0 3:b2c197ab:::smithi11731613-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.391928648000004, 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.120455968, 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 51 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.336 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924649+0000", 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924648+0000", 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924652+0000", 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.2350000000000001e-06 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924666+0000", 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4134999999999999e-05 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.338 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.924674+0000", 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2330000000000006e-06 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.926934+0000", 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0022601570000000001 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.927014+0000", 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.9526999999999999e-05 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953206+0000", 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026191849999999999 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:22.953257+0000", 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1428999999999998e-05 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.045104+0000", 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.091847236999999998 2024-09-16T06:49:41.340 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:124 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.284190983000002, 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016427374000000002, 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.341 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 124 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032386+0000", 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032384+0000", 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032895+0000", 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00051078400000000002 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032916+0000", 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1316999999999999e-05 2024-09-16T06:49:41.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032928+0000", 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1895999999999999e-05 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032987+0000", 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9503000000000003e-05 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033134+0000", 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014691899999999999 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033927+0000", 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00079267599999999997 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035097+0000", 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011696230000000001 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048736+0000", 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013639165999999999 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.048814+0000", 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7834000000000001e-05 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:125 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:41.346 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.283635524000005, 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016242589000000002, 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 125 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:41.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032942+0000", 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032941+0000", 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032949+0000", 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.3170000000000002e-06 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032958+0000", 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9530000000000005e-06 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032965+0000", 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1910000000000003e-06 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.349 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034475+0000", 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0015103320000000001 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034536+0000", 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0565e-05 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.034932+0000", 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000395843 2024-09-16T06:49:41.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035861+0000", 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00092933700000000002 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049108+0000", 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013246806999999999 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049184+0000", 2024-09-16T06:49:41.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6224999999999996e-05 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:126 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.283579192999994, 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031508340000000003, 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.352 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 126 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032998+0000", 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.032997+0000", 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033002+0000", 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3869999999999997e-06 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033025+0000", 2024-09-16T06:49:41.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3470999999999999e-05 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.033032+0000", 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1250000000000004e-06 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035124+0000", 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020913580000000002 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for rw locks", 2024-09-16T06:49:41.355 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.035198+0000", 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4068999999999998e-05 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049566+0000", 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014368485 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.049653+0000", 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.6933000000000003e-05 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.064506+0000", 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014853216000000001 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:219 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [copy-from ver 4] snapc 6=[6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 73.360360043, 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.031057211000000001, 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.357 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 219 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956217+0000", 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956214+0000", 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999962 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956234+0000", 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0275000000000002e-05 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956284+0000", 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0219999999999997e-05 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956309+0000", 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4930999999999999e-05 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956442+0000", 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013270999999999999 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956459+0000", 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7317999999999999e-05 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.960744+0000", 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0042850090000000002 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961009+0000", 2024-09-16T06:49:41.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026485300000000001 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.976584+0000", 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015575011999999999 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.981658+0000", 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050738470000000003 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987101+0000", 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0054435580000000003 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.987274+0000", 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017305700000000001 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:220 3.ds0 3:bbfb3f8a:::smithi11731613-42:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:41.363 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 73.360237584999993, 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.023931746, 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 220 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:41.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956340+0000", 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956338+0000", 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956351+0000", 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2849e-05 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956375+0000", 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4195000000000001e-05 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956391+0000", 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5352000000000001e-05 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956650+0000", 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025920099999999997 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.956669+0000", 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.9157999999999999e-05 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961818+0000", 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0051493709999999998 2024-09-16T06:49:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.961948+0000", 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012956 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979661+0000", 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017713421 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.979887+0000", 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00022550399999999999 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:27.980271+0000", 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00038454100000000002 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:316 3.ds0 3:b2c197ab:::smithi11731613-17:head [copy-from ver 4] snapc 8=[8,7,6,5,2] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:41.369 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 71.154562081999998, 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017020408000000001, 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 316 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162015+0000", 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162012+0000", 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162030+0000", 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8688000000000001e-05 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162067+0000", 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.6526000000000002e-05 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162083+0000", 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6061000000000001e-05 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162190+0000", 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000107351 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162511+0000", 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00032124399999999999 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.174275+0000", 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011763691999999999 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.176204+0000", 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019286329999999999 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.178962+0000", 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0027583099999999999 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.179036+0000", 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3456999999999998e-05 2024-09-16T06:49:41.374 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:317 3.ds0 3:b2c197ab:::smithi11731613-17:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 71.154476747000004, 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013682241, 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.375 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 317 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162100+0000", 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162106+0000", 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4620000000000001e-06 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162124+0000", 2024-09-16T06:49:41.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7706999999999998e-05 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162136+0000", 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2285e-05 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.162962+0000", 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000826123 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:41.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.163038+0000", 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.5371999999999996e-05 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175269+0000", 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01223169 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175524+0000", 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025445300000000003 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:30.175783+0000", 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025890899999999998 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:354 3.ds0 3:b273fc7d:::smithi11731613-1:head [read 0~966656,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 66.474039188999996, 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013614067000000001, 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.380 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 354 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.381 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842538+0000", 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842537+0000", 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842545+0000", 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2500000000000006e-06 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842562+0000", 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7512000000000001e-05 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.842573+0000", 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0976e-05 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844555+0000", 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001981919 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.844621+0000", 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5792999999999998e-05 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:34.856152+0000", 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011530883 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=96, reqid=osd.10.0:96, at_version=71'80, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.475842239000002, 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012334845000000001, 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.385 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840735+0000", 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840734+0000", 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840800+0000", 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.5324e-05 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840986+0000", 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000186701 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841011+0000", 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4372e-05 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841156+0000", 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000145749 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841215+0000", 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8845000000000003e-05 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853010+0000", 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011794392000000001 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.853070+0000", 2024-09-16T06:49:41.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0158000000000002e-05 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 71/32 ECSubWrite(tid=97, reqid=osd.10.0:97, at_version=71'82, trim_to=0'0, roll_forward_to=64'78))", 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.475769728000003, 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013582399, 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.389 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840807+0000", 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.840851+0000", 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4564999999999998e-05 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841065+0000", 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000213997 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.841082+0000", 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6833e-05 2024-09-16T06:49:41.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845856+0000", 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0047738549999999996 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.845879+0000", 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854322+0000", 2024-09-16T06:49:41.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0084429220000000003 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.854390+0000", 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7514000000000004e-05 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 71/37 ECSubWrite(tid=20, reqid=osd.5.0:20, at_version=71'17, trim_to=0'0, roll_forward_to=64'15))", 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:41.393 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.473415039000002, 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013186145, 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:41.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843162+0000", 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843211+0000", 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9116000000000002e-05 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843322+0000", 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000111142 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.843344+0000", 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2350000000000001e-05 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848942+0000", 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005598235 2024-09-16T06:49:41.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.848970+0000", 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7875999999999999e-05 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856277+0000", 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0073063049999999999 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:58.856348+0000", 2024-09-16T06:49:41.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.1741000000000003e-05 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.ds0 72/54 ECSubWrite(tid=156, reqid=osd.3.0:156, at_version=72'61, trim_to=0'0, roll_forward_to=71'59))", 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.478686127000003, 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013858534, 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.398 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837891+0000", 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837890+0000", 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837941+0000", 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0958999999999998e-05 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838125+0000", 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000184014 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838144+0000", 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8258999999999999e-05 2024-09-16T06:49:41.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838277+0000", 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000133452 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838303+0000", 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6038999999999999e-05 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851693+0000", 2024-09-16T06:49:41.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013389803 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.851750+0000", 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6641000000000002e-05 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as1 94/32 ECSubWrite(tid=260, reqid=client.4511.0:852, at_version=94'198, trim_to=0'0, roll_forward_to=92'196))", 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.373293018999998, 2024-09-16T06:49:41.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014393538000000001, 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943284+0000", 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.403 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943283+0000", 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943342+0000", 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8780999999999998e-05 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943490+0000", 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014785699999999999 2024-09-16T06:49:41.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.943516+0000", 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5681999999999999e-05 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944924+0000", 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00140785 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944950+0000", 2024-09-16T06:49:41.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6622e-05 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.957559+0000", 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012609120999999999 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.957678+0000", 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000118449 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.bs5 94/77 ECSubWrite(tid=160, reqid=osd.13.0:160, at_version=94'29, trim_to=0'0, roll_forward_to=92'27))", 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.372395945000001, 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01233383, 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944181+0000", 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944271+0000", 2024-09-16T06:49:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9946999999999998e-05 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944400+0000", 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012965399999999999 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944414+0000", 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3454e-05 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.409 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944586+0000", 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000171893 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944603+0000", 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6923e-05 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956440+0000", 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011837640999999999 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956515+0000", 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4670000000000002e-05 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s1 94/32 ECSubWrite(tid=164, reqid=osd.10.0:164, at_version=94'136, trim_to=0'0, roll_forward_to=92'134))", 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 19.372237333000001, 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012091431999999999, 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.411 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.412 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944340+0000", 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944367+0000", 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7620999999999999e-05 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944472+0000", 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105237 2024-09-16T06:49:41.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944482+0000", 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7540000000000001e-06 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944702+0000", 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000220034 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.944716+0000", 2024-09-16T06:49:41.414 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3465000000000001e-05 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956349+0000", 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011633596 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.956431+0000", 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.2027000000000006e-05 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.6s5 104/102 ECSubWrite(tid=319, reqid=osd.3.0:319, at_version=104'34, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 9.4856458969999995, 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014382463, 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830931+0000", 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.830987+0000", 2024-09-16T06:49:41.417 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6462000000000003e-05 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.831207+0000", 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021981799999999999 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.831229+0000", 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2103000000000002e-05 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.418 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.839034+0000", 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0078046870000000003 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.839080+0000", 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.6186000000000001e-05 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.845257+0000", 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0061766820000000002 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.419 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.845314+0000", 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.6975000000000003e-05 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.420 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:41.421 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T06:49:41.546 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:41.546 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:41.546 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:41.546 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:41.560 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T06:49:41.682 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:41.682 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:41.683 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:41.683 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:41.683 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:41.696 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 35/32 ECSubWrite(tid=35, reqid=client.4511.0:152, at_version=35'10, trim_to=0'0, roll_forward_to=35'5))", 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:41.823 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.742724581000004, 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.44133388099999998, 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077840+0000", 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077839+0000", 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.077850+0000", 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0505e-05 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515878+0000", 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.43802848300000002 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515883+0000", 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1780000000000002e-06 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515923+0000", 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9866000000000001e-05 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.826 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.515930+0000", 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.224e-06 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519096+0000", 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0031664449999999999 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.519174+0000", 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.7590999999999996e-05 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.827 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:163 3.cs0 3:31c9f1c0:::smithi11731613-46:head [copy-from ver 2 in=54b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.308482890999997, 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.80778171600000004, 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 163 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.828 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512081+0000", 2024-09-16T06:49:41.829 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512085+0000", 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8870000000000002e-06 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512091+0000", 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3539999999999996e-06 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.512096+0000", 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8029999999999996e-06 2024-09-16T06:49:41.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513808+0000", 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001712199 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.513857+0000", 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8649000000000001e-05 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.524232+0000", 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010375422 2024-09-16T06:49:41.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.527189+0000", 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002956939 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319694+0000", 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79250512900000003 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319863+0000", 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168796 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.293855821999998, 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79400328899999995, 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.833 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526708+0000", 2024-09-16T06:49:41.834 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.526711+0000", 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.18e-06 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319361+0000", 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.79264922400000004 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319384+0000", 2024-09-16T06:49:41.835 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3442e-05 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.319508+0000", 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000123683 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:24.320712+0000", 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0012041 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.836 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 35/32 ECSubWriteReply(tid=34, last_complete=35'10, committed=1, applied=1))", 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 78.291880878000001, 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.394952786, 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.837 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528683+0000", 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.528687+0000", 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0810000000000004e-06 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.838 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923559+0000", 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.39487182199999998 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923572+0000", 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3406e-05 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923624+0000", 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.1989e-05 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.839 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:23.923636+0000", 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1990999999999999e-05 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 49/37 ECSubWriteReply(tid=97, last_complete=49'46, committed=1, applied=1))", 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 64.975764717999994, 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15568405900000001, 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.840 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844800+0000", 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.841 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844798+0000", 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.844807+0000", 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.8319999999999995e-06 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.998993+0000", 2024-09-16T06:49:41.842 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15418552099999999 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999006+0000", 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3069e-05 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.999149+0000", 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000142965 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000484+0000", 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013350300000000001 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 49/37 ECSubWriteReply(tid=98, last_complete=49'26, committed=1, applied=1))", 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 64.970053796000002, 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15060995699999999, 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850511+0000", 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850509+0000", 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.850523+0000", 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.4219000000000001e-05 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000937+0000", 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.15041373399999999 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.000960+0000", 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3768e-05 2024-09-16T06:49:41.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001073+0000", 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000112218 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:37.001120+0000", 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7936999999999999e-05 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.847 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:428 3.cs0 3:31c9f1c0:::smithi11731613-46:head [delete] snapc 13=[13,11,e,d,c,b,8,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 60.965662532000003, 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.62638514099999998, 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 428 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:41.848 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854902+0000", 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854899+0000", 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.849 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854915+0000", 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5262999999999999e-05 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854948+0000", 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3005e-05 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.854965+0000", 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7112000000000002e-05 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.850 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855070+0000", 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105198 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.855081+0000", 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1308000000000001e-05 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478008+0000", 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.622927122 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.851 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478196+0000", 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000187534 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.481287+0000", 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0030910360000000001 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.852 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(osd.5.22:1 3.fs0 3:fa1c58b4:::smithi11731613-7:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 60.951884292000003, 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.61158470300000001, 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.853 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868680+0000", 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868678+0000", 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868690+0000", 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2336e-05 2024-09-16T06:49:41.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868722+0000", 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2122000000000002e-05 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868744+0000", 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1855e-05 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868876+0000", 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013210300000000001 2024-09-16T06:49:41.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:40.868888+0000", 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2053e-05 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478066+0000", 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.60917769200000005 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.478207+0000", 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000140743 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:41.480265+0000", 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0020579219999999998 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:583 3.fs0 3:fa1c58b4:::smithi11731613-7:head [append 0~0] snapc 24=[24,22,21,20,1f,1d,1c,19,18,15,11] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:41.857 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.761232829999997, 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41724843499999997, 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 583 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.858 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059331+0000", 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.859 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059336+0000", 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5140000000000001e-06 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059357+0000", 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0602e-05 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.059374+0000", 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.7640000000000001e-05 2024-09-16T06:49:41.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062783+0000", 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.003409081 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.062871+0000", 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7188999999999995e-05 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.063266+0000", 2024-09-16T06:49:41.861 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00039578200000000001 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.064116+0000", 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00084932599999999997 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476470+0000", 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41235417099999999 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476580+0000", 2024-09-16T06:49:41.862 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011006999999999999 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 72/37 ECSubWriteReply(tid=209, last_complete=72'94, committed=1, applied=1))", 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.755004925000001, 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41187659199999999, 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:41.863 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065559+0000", 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.065564+0000", 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2839999999999997e-06 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476221+0000", 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.41065675400000001 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476236+0000", 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5509e-05 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.865 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.476326+0000", 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.9337000000000002e-05 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.477436+0000", 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011104210000000001 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.866 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.1s1 81/77 ECSubWrite(tid=85, reqid=client.4511.0:732, at_version=0'0, trim_to=0'0, roll_forward_to=81'47))", 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 32.513940853999998, 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.38681583800000002, 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306623+0000", 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306621+0000", 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.306649+0000", 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7733e-05 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.868 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692262+0000", 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.385613345 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692278+0000", 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5642e-05 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692421+0000", 2024-09-16T06:49:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014278699999999999 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.692441+0000", 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0089e-05 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693380+0000", 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00093900299999999999 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:09.693439+0000", 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9342000000000002e-05 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDScrubReserve(3.as6 REQUEST e98)", 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:26.350367+0000", 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 15.470197527, 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.42875901900000002, 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350367+0000", 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350367+0000", 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350365+0000", 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:41.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.350379+0000", 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.3655999999999999e-05 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.778878+0000", 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.42849893700000002 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.778892+0000", 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.398e-05 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.779028+0000", 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000136587 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.779057+0000", 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8615999999999998e-05 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.874 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:26.779126+0000", 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.8955000000000006e-05 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:934 3.fs0 3:fbae4168:::smithi11731613-21:head [create,setxattr tks`btPmsWciQOeYkTqWf`O^jQPan_cobnZerePO (1333) in=40b,setxattr \\]Q`aZUXRU_mQP_R_VijPtfgkatRsaNXgktfbort (102) in=40b,setxattr cnZX]a[_P_erokmkXY^dYWhTet_fSmmt`m\\nme`V (6244) in=40b,setxattr UkjTkbhYhVmTa[PPbrfXd\\VWd`QTYWbYamfUUmn_ (839) in=40b,setxattr nsi[hrP\\iPq\\jjZg^UtgbYkXj^dq\\fmSgdhbetoQ (1086) in=40b,setxattr [tgkYkY`fSah\\ahgj]RmkptX\\]h`T[gc]eag\\`b_ (2876) in=40b,setxattr QaOge_]eoPirtZlWSQ^\\gN_bgQOnfO]WdoqeracP (18032) in=40b,setxattr mi[nPcPgXbUbtkkjTY_pghrlZY_gk]Uk`_lmeTR\\ (1646) in=40b,setxattr oZtfSNa_W_re_`TbbYt[aPepifks`OqdQXYQa]od (2876) in=40b,setxattr QaYqal[djghoh`jsarPORqq_NqPekbXlacljdX\\f (350) in=40b,setxattr SiYahrpPZgdoSYbjtoXiggitTbphkZYhlW]ORlsa (523) in=40b,setxattr fPlNX^qbScs\\dcoWYSOmN_ZPPc_`rd\\TcQbjRod] (349) in=40b,setxattr PhkaTtafYaZ^nX]`bbR^QkYXepSSfjeqaYpgp]sa (12488) in=40b,setxattr aQ]`dlhjNXpgabS]eeS[j^Qhackqi`NZap[nUla\\ (6736) in=40b,setxattr QbfiUncStonfq^rXh`RaOPPmaOtpUpPX_]ob_\\\\t (6244) in=40b,setxattr UPURgp\\idZifafiaW[dX^WQSa_mZ]lsORaprWQYn (875) in=40b,setxattr N]e]t\\kgiWZilkcVqpgg (208) in=20b,setxattr YknjWPR]WkZl[e\\OiP\\[QYNb_RjhrsddS[RiOW[a (14664) in=40b,setxattr iWnNoigT`Pa_fY[[dPqrrPaTl_ptOPchg\\]theNm (1683) in=40b,setxattr RPrnabOkbfqbSYdbXaTf_gh`S^RgP^ohg`Rddbqn (805) in=40b] snapc 44=[44,43,41,40,3e,3c,3a,33,31,28,1c,19,18] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:31.023596+0000", 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 10.796968492, 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.8449128350000001, 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.875 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 934 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023596+0000", 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023596+0000", 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023593+0000", 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999971 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023747+0000", 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000154363 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.877 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023786+0000", 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8636000000000001e-05 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023810+0000", 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4006e-05 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023897+0000", 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7430999999999997e-05 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.878 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.023903+0000", 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3550000000000004e-06 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836210+0000", 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.81230698 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836213+0000", 2024-09-16T06:49:41.879 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.3299999999999999e-06 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.858868+0000", 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.022654876 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.859317+0000", 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000449016 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.880 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.861268+0000", 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001951604 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.863681+0000", 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024128109999999999 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.868047+0000", 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.004366188 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.881 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.868509+0000", 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00046132200000000001 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.7s6 104/77 ECSubWrite(tid=226, reqid=osd.13.0:226, at_version=104'159, trim_to=0'0, roll_forward_to=104'157))", 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:31.869610+0000", 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 9.9509545979999992, 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.70657596600000006, 2024-09-16T06:49:41.882 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869610+0000", 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869610+0000", 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.883 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869608+0000", 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:31.869656+0000", 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.7533e-05 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573471+0000", 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.70381538499999996 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.884 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573487+0000", 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5498999999999998e-05 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573632+0000", 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014505 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.573652+0000", 2024-09-16T06:49:41.885 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0440000000000001e-05 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.576125+0000", 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.002472756 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.576186+0000", 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.0649000000000002e-05 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:946 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~0,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835018+0000", 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9855459999999994, 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.032809079, 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 946 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.887 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835018+0000", 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835018+0000", 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835016+0000", 2024-09-16T06:49:41.888 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835024+0000", 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.6569999999999997e-06 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835049+0000", 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4983000000000001e-05 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835059+0000", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.059e-05 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836243+0000", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0011832559999999999 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836243+0000", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8500000000000001e-07 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.863780+0000", 2024-09-16T06:49:41.890 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0275364920000001 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for missing object", 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.863953+0000", 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017281100000000001 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.867709+0000", 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0037559519999999999 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.867827+0000", 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011885200000000001 2024-09-16T06:49:41.891 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:947 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~0,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9854958509999996, 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0342719360000001, 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.892 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 947 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.893 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835068+0000", 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835072+0000", 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.1180000000000002e-06 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835079+0000", 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4330000000000001e-06 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.894 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835084+0000", 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.9969999999999998e-06 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836346+0000", 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001262958 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836348+0000", 2024-09-16T06:49:41.895 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.468e-06 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864122+0000", 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.027773923 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for missing object", 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864170+0000", 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8010999999999998e-05 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869237+0000", 2024-09-16T06:49:41.896 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.005067111 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869340+0000", 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000103415 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:948 3.fs0 3:f3961c35:::smithi11731613-9:head [read 0~0,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835089+0000", 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9854757450000005, 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0344568949999999, 2024-09-16T06:49:41.897 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 948 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835089+0000", 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835089+0000", 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835088+0000", 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835091+0000", 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0350000000000002e-06 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835098+0000", 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.2520000000000002e-06 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835104+0000", 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4029999999999998e-06 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836442+0000", 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0013385960000000001 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.900 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836447+0000", 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.8930000000000004e-06 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864211+0000", 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0277632809999999 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for missing object", 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864301+0000", 2024-09-16T06:49:41.901 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0840000000000004e-05 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869379+0000", 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0050770750000000003 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.869545+0000", 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000166921 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:951 3.fs0 3:fa1c58b4:::smithi11731613-7:head [copy-from ver 32 in=55b] snapc 44=[44,43,41,40,3e,3c,3a,31,28,19,18] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835109+0000", 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9854557509999999, 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0163427559999998, 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 951 2024-09-16T06:49:41.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835109+0000", 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835109+0000", 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835108+0000", 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835111+0000", 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.2310000000000001e-06 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835120+0000", 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.5569999999999996e-06 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835126+0000", 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.5910000000000003e-06 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836529+0000", 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.001403219 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836530+0000", 2024-09-16T06:49:41.906 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.511e-06 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864339+0000", 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0278086719999999 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864498+0000", 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015914999999999999 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:41.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.914456+0000", 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.049957930999999997 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.920830+0000", 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0063740689999999996 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.850851+0000", 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.93002128500000003 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.908 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.851451+0000", 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00059985699999999997 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:952 3.fs0 3:fa1c58b4:::smithi11731613-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e105)", 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:32.835130+0000", 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 8.9854345070000008, 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.084286708, 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.909 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 952 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835130+0000", 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835130+0000", 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835129+0000", 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835134+0000", 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5759999999999999e-06 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.911 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835140+0000", 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4849999999999997e-06 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.835147+0000", 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6370000000000002e-06 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836576+0000", 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0014293450000000001 2024-09-16T06:49:41.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for peered", 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:32.836579+0000", 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1210000000000001e-06 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864677+0000", 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.028098347 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting for blocked object", 2024-09-16T06:49:41.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.864808+0000", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013064399999999999 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.918901+0000", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.054093585999999999 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.919074+0000", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017242200000000001 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:33.919417+0000", 2024-09-16T06:49:41.914 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00034298500000000002 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDPGPushReply(3.fs0 106/105 [PushReplyOp(3:f943c919:::smithi11731613-41:35),PushReplyOp(3:f943c919:::smithi11731613-41:head),PushReplyOp(3:fbae4168:::smithi11731613-21:44),PushReplyOp(3:fbae4168:::smithi11731613-21:head)])", 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:34.509296+0000", 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 7.3112680360000004, 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.329643774, 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.915 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.509296+0000", 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.509296+0000", 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.509294+0000", 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999976 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.916 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.509310+0000", 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5622999999999999e-05 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.837907+0000", 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.32859719799999998 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.837918+0000", 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.1131000000000001e-05 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.917 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.838229+0000", 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031114700000000001 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:34.838940+0000", 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00071096199999999999 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:41.918 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:41.920 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-16T06:49:42.046 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:42.047 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:42.047 INFO:teuthology.orchestra.run.smithi117.stdout: "num_ops": 0 2024-09-16T06:49:42.047 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:42.061 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-16T06:49:42.061 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-16T06:49:42.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T06:49:42.177 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [], 2024-09-16T06:49:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "complaint_time": 30, 2024-09-16T06:49:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "num_blocked_ops": 0 2024-09-16T06:49:42.177 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:42.191 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-16T06:49:42.318 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-16T06:49:42.318 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 20, 2024-09-16T06:49:42.318 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 600, 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "ops": [ 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:266 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 73.362144021000006, 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014774862999999999, 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 266 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.319 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953829+0000", 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.320 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953825+0000", 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953846+0000", 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0171999999999999e-05 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953889+0000", 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.3284000000000001e-05 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.321 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953913+0000", 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4071e-05 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954055+0000", 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00014240699999999999 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "op must wait for map", 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954076+0000", 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.0678000000000001e-05 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.322 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961148+0000", 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0070716959999999997 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.961402+0000", 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025470100000000003 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.968604+0000", 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0072013340000000002 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.323 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:267 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 73.362027808999997, 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015485804000000001, 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 267 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.324 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953945+0000", 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953944+0000", 2024-09-16T06:49:42.325 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953973+0000", 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9581e-05 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.953995+0000", 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1685e-05 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954011+0000", 2024-09-16T06:49:42.326 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5862000000000002e-05 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954223+0000", 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00021187199999999999 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954233+0000", 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.0789e-05 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962540+0000", 2024-09-16T06:49:42.327 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0083068819999999998 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.962710+0000", 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017007799999999999 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.969431+0000", 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0067204250000000004 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:268 3.5s0 3:af79444c:::smithi11731613-29:7 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 73.361947561999997, 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.30266284300000001, 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 268 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.329 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954025+0000", 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954024+0000", 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.330 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954032+0000", 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.4290000000000004e-06 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954050+0000", 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.8332e-05 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954063+0000", 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.2955999999999999e-05 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.331 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954379+0000", 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00031616000000000001 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "waiting_for_map not empty", 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.954389+0000", 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.3959999999999995e-06 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963428+0000", 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0090396959999999998 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:28.963723+0000", 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00029431800000000001 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:29.256688+0000", 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.29296564200000003 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.333 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 49/32 ECSubWrite(tid=52, reqid=osd.14.0:52, at_version=49'38, trim_to=0'0, roll_forward_to=44'34))", 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 65.480685373, 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014247134, 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.334 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835288+0000", 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835286+0000", 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835349+0000", 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.3258999999999999e-05 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.335 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835459+0000", 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000110097 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835480+0000", 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.1424999999999999e-05 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835600+0000", 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00011922 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.336 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.835623+0000", 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3144999999999999e-05 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849454+0000", 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013830779 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:36.849535+0000", 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.1021999999999998e-05 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.337 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 61/32 ECSubWrite(tid=83, reqid=osd.14.0:83, at_version=61'72, trim_to=0'0, roll_forward_to=59'68))", 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 53.482400196, 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012777472999999999, 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833573+0000", 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833571+0000", 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833631+0000", 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9917999999999998e-05 2024-09-16T06:49:42.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833761+0000", 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013055799999999999 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.833784+0000", 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2714999999999999e-05 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.340 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835926+0000", 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0021422720000000002 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.835943+0000", 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6889e-05 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846281+0000", 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010337641999999999 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.341 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.846350+0000", 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9315999999999998e-05 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 61/32 ECSubWrite(tid=85, reqid=osd.10.0:85, at_version=61'70, trim_to=0'0, roll_forward_to=59'66))", 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 53.477518463000003, 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013138069, 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838454+0000", 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838452+0000", 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:42.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.838506+0000", 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3594999999999998e-05 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839270+0000", 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00076359400000000003 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.344 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839293+0000", 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3006e-05 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839420+0000", 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000127355 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.839446+0000", 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.6225999999999999e-05 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.345 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851525+0000", 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012079196 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:48.851592+0000", 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7156999999999994e-05 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=101, reqid=osd.10.0:101, at_version=72'86, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:42.346 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.478505269000003, 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017459348, 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837468+0000", 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.347 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837467+0000", 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837538+0000", 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.0541000000000001e-05 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837664+0000", 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012650899999999999 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.348 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837687+0000", 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2759999999999999e-05 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839677+0000", 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0019901300000000001 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.839707+0000", 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.0049e-05 2024-09-16T06:49:42.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854862+0000", 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015155366 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854927+0000", 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4629999999999996e-05 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.350 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=157, reqid=osd.3.0:157, at_version=72'123, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.478192315999998, 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.027234286, 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837781+0000", 2024-09-16T06:49:42.351 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837780+0000", 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837839+0000", 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8801000000000002e-05 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837959+0000", 2024-09-16T06:49:42.352 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000120436 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.837982+0000", 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3156e-05 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838118+0000", 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000135673 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.353 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.838143+0000", 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4899e-05 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.864948+0000", 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.026805539999999999 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865015+0000", 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.6501000000000006e-05 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.354 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 72/32 ECSubWrite(tid=102, reqid=osd.10.0:102, at_version=72'88, trim_to=0'0, roll_forward_to=71'84))", 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.475381331999998, 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.018351104, 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840592+0000", 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840590+0000", 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.356 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840640+0000", 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0631999999999999e-05 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840728+0000", 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 8.7933e-05 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840750+0000", 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2022999999999999e-05 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.357 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840864+0000", 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113423 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.840889+0000", 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.5092999999999998e-05 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858883+0000", 2024-09-16T06:49:42.358 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017994231999999999 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.858943+0000", 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.9587999999999999e-05 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.as5 72/32 ECSubWrite(tid=158, reqid=osd.3.0:158, at_version=72'125, trim_to=0'0, roll_forward_to=71'121))", 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.473740194999998, 2024-09-16T06:49:42.359 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.022836532999999999, 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842233+0000", 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842231+0000", 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842295+0000", 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.4238000000000004e-05 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842418+0000", 2024-09-16T06:49:42.361 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00012244299999999999 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.842441+0000", 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3606000000000001e-05 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850726+0000", 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082850219999999995 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.362 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850750+0000", 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3822e-05 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865034+0000", 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014283563000000001 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865069+0000", 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.5712e-05 2024-09-16T06:49:42.363 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=65, reqid=osd.12.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.470417169999997, 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017333787, 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:42.364 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845556+0000", 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845555+0000", 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.365 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845609+0000", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.4284999999999999e-05 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845766+0000", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015658100000000001 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845789+0000", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2866000000000001e-05 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845942+0000", 2024-09-16T06:49:42.366 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00015326200000000001 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.845969+0000", 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.7296000000000001e-05 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862827+0000", 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.016857351999999999 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.367 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862890+0000", 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.2997999999999996e-05 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 72/32 ECSubWrite(tid=66, reqid=osd.12.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.468725683999999, 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017965090999999999, 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.368 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847247+0000", 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847246+0000", 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999986 2024-09-16T06:49:42.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847298+0000", 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.2677999999999999e-05 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847412+0000", 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000113403 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.847437+0000", 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4973e-05 2024-09-16T06:49:42.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857022+0000", 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0095850889999999998 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.857045+0000", 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3796000000000001e-05 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865168+0000", 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0081221260000000003 2024-09-16T06:49:42.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865212+0000", 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4691999999999998e-05 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=108, reqid=osd.14.0:108, at_version=72'100, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.465626671999999, 2024-09-16T06:49:42.372 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015047932999999999, 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850346+0000", 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.373 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850339+0000", 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999928 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850413+0000", 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 7.3508000000000002e-05 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850503+0000", 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.0303999999999995e-05 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.374 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850526+0000", 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2595999999999998e-05 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850694+0000", 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000168367 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.850718+0000", 2024-09-16T06:49:42.375 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.44e-05 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865344+0000", 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014625658999999999 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.865394+0000", 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0074999999999997e-05 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/32 ECSubWrite(tid=109, reqid=osd.14.0:109, at_version=72'102, trim_to=0'0, roll_forward_to=71'98))", 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.461979104000001, 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.013551083, 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853994+0000", 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.853992+0000", 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999981 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.378 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854050+0000", 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.8495000000000002e-05 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854252+0000", 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00020159999999999999 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.854296+0000", 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.4438999999999999e-05 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.379 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862592+0000", 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0082959690000000003 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.862662+0000", 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.9210999999999999e-05 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867494+0000", 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0048324029999999999 2024-09-16T06:49:42.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.867545+0000", 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.0838999999999999e-05 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:559 3.5s0 3:af79444c:::smithi11731613-29:1d [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 42.360992946000003, 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.014695526, 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.381 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "started", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 559 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954980+0000", 2024-09-16T06:49:42.382 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954979+0000", 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954984+0000", 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.5310000000000003e-06 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.383 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.954994+0000", 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.7249999999999992e-06 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.955000+0000", 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 6.7569999999999998e-06 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967068+0000", 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012068096 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:42.384 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.967239+0000", 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00017012400000000001 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:48:59.969675+0000", 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0024368660000000002 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/32 ECSubWrite(tid=192, reqid=osd.6.0:192, at_version=73'33, trim_to=0'0, roll_forward_to=72'29))", 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:42.385 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.467186861000002, 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.017679872999999999, 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848786+0000", 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.386 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848785+0000", 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.848820+0000", 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.4931999999999997e-05 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849087+0000", 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00026683499999999999 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.387 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.849103+0000", 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.6813e-05 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855706+0000", 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066022570000000003 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855730+0000", 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4661999999999999e-05 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.388 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866426+0000", 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.010695893999999999 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866466+0000", 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.9632999999999998e-05 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.389 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.0s2 73/32 ECSubWrite(tid=78, reqid=osd.12.0:78, at_version=73'56, trim_to=0'0, roll_forward_to=72'54))", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.465060958999999, 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.015506908999999999, 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850912+0000", 2024-09-16T06:49:42.390 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850911+0000", 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.850952+0000", 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4.0722000000000001e-05 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851117+0000", 2024-09-16T06:49:42.391 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000164599 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.851132+0000", 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 1.5255e-05 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854555+0000", 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0034234320000000001 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.392 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854579+0000", 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.3655999999999998e-05 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866327+0000", 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.011748364000000001 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866419+0000", 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 9.1624000000000002e-05 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.3s3 73/32 ECSubWrite(tid=107, reqid=osd.10.0:107, at_version=73'90, trim_to=0'0, roll_forward_to=72'88))", 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 41.461808488000003, 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012199229000000001, 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.394 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854164+0000", 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999995 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854217+0000", 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.3168000000000003e-05 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.395 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854327+0000", 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000109833 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.854349+0000", 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2296000000000001e-05 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855253+0000", 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00090369500000000004 2024-09-16T06:49:42.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.855283+0000", 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.9884999999999999e-05 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866242+0000", 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.01095932 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:00.866364+0000", 2024-09-16T06:49:42.397 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000121542 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "osd_op(client.4511.0:705 3.5s0 3:af79444c:::smithi11731613-29:head [append 0~524288] snapc 2b=[2b,2a,29,28,24,22,21,20,1f,1d,1c,19,18,15] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 36.282236914000002, 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012998625, 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "client_info": { 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "client": "client.4511", 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "client_addr": "172.21.15.117:0/2087889830", 2024-09-16T06:49:42.398 INFO:teuthology.orchestra.run.smithi117.stdout: "tid": 705 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033736+0000", 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.033733+0000", 2024-09-16T06:49:42.399 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.9999967 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034535+0000", 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00080219500000000001 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034573+0000", 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.8473000000000002e-05 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.400 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034605+0000", 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 3.1776999999999997e-05 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.034755+0000", 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000149641 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "started", 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.035008+0000", 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00025373600000000001 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.401 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.039658+0000", 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0046500159999999999 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046264+0000", 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.0066053680000000004 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "commit_sent", 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046596+0000", 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000331983 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.402 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:06.046735+0000", 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00013881599999999999 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "description": "MOSDECSubOpWrite(3.2s4 94/32 ECSubWrite(tid=165, reqid=osd.14.0:165, at_version=94'158, trim_to=0'0, roll_forward_to=92'156))", 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "age": 20.380283127999999, 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012800693, 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type_data": { 2024-09-16T06:49:42.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flag_point": "reached pg", 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "initiated", 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "throttled", 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935690+0000", 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "header_read", 2024-09-16T06:49:42.404 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935689+0000", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 4294967295.999999 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "all_read", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935746+0000", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 5.7460999999999998e-05 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "dispatched", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935852+0000", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000105815 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "queued_for_pg", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.935875+0000", 2024-09-16T06:49:42.405 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.2886000000000001e-05 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "reached_pg", 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936046+0000", 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.000170929 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_started", 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.936070+0000", 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 2.4746e-05 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "sub_op_committed", 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.948119+0000", 2024-09-16T06:49:42.406 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.012048432 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: "event": "done", 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: "time": "2024-09-16T06:49:21.948490+0000", 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: "duration": 0.00037153400000000002 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-16T06:49:42.407 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-16T06:49:42.637 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-16T06:49:42.638 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-16T06:49:42.638 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T06:49:42.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T06:49:42.641 INFO:tasks.ceph.osd.5:Started 2024-09-16T06:49:42.641 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:49:42.767 INFO:teuthology.orchestra.run.smithi107.stderr:no valid command found; 10 closest matches: 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:0 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:1 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:2 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:abort 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:assert 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:config diff 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:config diff get 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:config get 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:config help [] 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:config set ... 2024-09-16T06:49:42.768 INFO:teuthology.orchestra.run.smithi107.stderr:admin_socket: invalid command 2024-09-16T06:49:42.770 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T06:49:42.770 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-16T06:49:43.065 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-16T06:49:43.250 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:43.249+0000 7f42933617c0 -1 Falling back to public interface 2024-09-16T06:49:47.040 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:47.036+0000 7f42933617c0 -1 osd.5 82 log_to_monitors true 2024-09-16T06:49:47.146 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-09-16T06:49:47.144+0000 7f428b130640 -1 osd.5 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T06:49:47.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T06:49:47.903 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-16T06:49:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-16T06:49:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-16T06:49:47.903 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-16T06:49:47.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T06:49:48.072 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a: {} 2024-09-16T06:49:48.073 INFO:teuthology.orchestra.run.smithi100.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T06:49:48.090 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b: {} 2024-09-16T06:49:48.091 INFO:teuthology.orchestra.run.smithi100.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T06:49:48.109 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c: {} 2024-09-16T06:49:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T06:49:48.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T06:49:49.125 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T06:49:49.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T06:49:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T06:49:50.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T06:49:51.134 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T06:49:51.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T06:49:52.143 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T06:49:52.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-16T06:49:53.157 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.8 to 1 (10000) 2024-09-16T06:49:53.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-16T06:49:54.167 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-16T06:49:54.193 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-16T06:49:55.178 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.12 to 1 (10000) 2024-09-16T06:49:55.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-09-16T06:49:56.186 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-09-16T06:49:56.209 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T06:49:57.196 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T06:49:57.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T06:49:58.206 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T06:49:58.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-16T06:49:59.225 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.9 to 1 (10000) 2024-09-16T06:49:59.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-16T06:50:00.233 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-16T06:50:00.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-16T06:50:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.13 to 1 (10000) 2024-09-16T06:50:01.260 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-09-16T06:50:02.246 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-09-16T06:50:02.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T06:50:03.259 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T06:50:03.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T06:50:04.270 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T06:50:04.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T06:50:05.280 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T06:50:05.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T06:50:06.291 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T06:50:06.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-16T06:50:07.303 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.10 to 1 (10000) 2024-09-16T06:50:07.325 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-16T06:50:08.315 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-16T06:50:08.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-16T06:50:09.326 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.14 to 1 (10000) 2024-09-16T06:50:09.349 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-09-16T06:50:10.342 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-09-16T06:50:10.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T06:50:11.344 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T06:50:11.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T06:50:12.353 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T06:50:12.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T06:50:13.365 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T06:50:13.388 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T06:50:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T06:50:14.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-16T06:50:15.388 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.11 to 1 (10000) 2024-09-16T06:50:15.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-16T06:50:16.398 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-16T06:50:16.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-16T06:50:17.409 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.15 to 1 (10000) 2024-09-16T06:50:17.434 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-09-16T06:50:18.419 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-09-16T06:50:18.447 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T06:50:19.435 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T06:50:19.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T06:50:20.449 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T06:50:20.470 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T06:50:20.470 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T06:50:20.470 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T06:50:20.471 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T06:50:20.471 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T06:50:20.471 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:50:20.775 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:20.775 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":147,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:50:20.436813+0000","last_up_change":"2024-09-16T06:49:48.103739+0000","last_in_change":"2024-09-16T06:49:04.862161+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T06:48:08.489111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":82,"up_from":115,"up_thru":37,"down_at":83,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6812","nonce":1072021825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6816","nonce":1072021825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6823","nonce":1072021825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6820","nonce":1072021825}]},"public_addr":"172.21.15.107:6812/1072021825","cluster_addr":"172.21.15.107:6816/1072021825","heartbeat_back_addr":"172.21.15.107:6823/1072021825","heartbeat_front_addr":"172.21.15.107:6820/1072021825","state":["exists","up"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.319627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:31.889358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.542092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:28.191679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:17.579785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:48.020278+0000","dead_epoch":83},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:21.476717+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:11.670056+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:19.668799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:25.819931+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:50:06.725344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.647481+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.898585+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.022637+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.810301+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:33.012714+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":21,"length":1},{"begin":28,"length":2},{"begin":31,"length":3},{"begin":41,"length":4},{"begin":46,"length":1},{"begin":51,"length":1},{"begin":53,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:50:20.794 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T06:50:20.794 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:50:21.090 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:21.090 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":147,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:50:20.436813+0000","last_up_change":"2024-09-16T06:49:48.103739+0000","last_in_change":"2024-09-16T06:49:04.862161+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T06:48:08.489111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":82,"up_from":115,"up_thru":37,"down_at":83,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6812","nonce":1072021825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6816","nonce":1072021825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6823","nonce":1072021825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6820","nonce":1072021825}]},"public_addr":"172.21.15.107:6812/1072021825","cluster_addr":"172.21.15.107:6816/1072021825","heartbeat_back_addr":"172.21.15.107:6823/1072021825","heartbeat_front_addr":"172.21.15.107:6820/1072021825","state":["exists","up"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.319627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:31.889358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.542092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:28.191679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:17.579785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:48.020278+0000","dead_epoch":83},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:21.476717+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:11.670056+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:19.668799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:25.819931+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:50:06.725344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.647481+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.898585+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.022637+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.810301+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:33.012714+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":21,"length":1},{"begin":28,"length":2},{"begin":31,"length":3},{"begin":41,"length":4},{"begin":46,"length":1},{"begin":51,"length":1},{"begin":53,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:50:21.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T06:50:21.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T06:50:21.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T06:50:21.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T06:50:21.110 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T06:50:21.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T06:50:21.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T06:50:21.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T06:50:21.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-16T06:50:21.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-16T06:50:21.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-16T06:50:21.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-16T06:50:21.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-16T06:50:21.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-16T06:50:21.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-16T06:50:21.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-16T06:50:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:98784247838 2024-09-16T06:50:21.511 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-16T06:50:21.549 INFO:teuthology.orchestra.run.smithi100.stdout:98784247840 2024-09-16T06:50:21.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T06:50:21.558 INFO:teuthology.orchestra.run.smithi100.stdout:94489280542 2024-09-16T06:50:21.559 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T06:50:21.615 INFO:teuthology.orchestra.run.smithi100.stdout:94489280543 2024-09-16T06:50:21.615 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T06:50:21.620 INFO:teuthology.orchestra.run.smithi100.stdout:98784247838 2024-09-16T06:50:21.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-16T06:50:21.634 INFO:teuthology.orchestra.run.smithi100.stdout:94489280545 2024-09-16T06:50:21.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T06:50:21.644 INFO:teuthology.orchestra.run.smithi100.stdout:98784247839 2024-09-16T06:50:21.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T06:50:21.649 INFO:teuthology.orchestra.run.smithi100.stdout:98784247838 2024-09-16T06:50:21.649 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T06:50:21.657 INFO:teuthology.orchestra.run.smithi100.stdout:98784247837 2024-09-16T06:50:21.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-16T06:50:21.659 INFO:teuthology.orchestra.run.smithi100.stdout:90194313247 2024-09-16T06:50:21.660 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-16T06:50:21.663 INFO:teuthology.orchestra.run.smithi100.stdout:493921239048 2024-09-16T06:50:21.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T06:50:21.666 INFO:teuthology.orchestra.run.smithi100.stdout:98784247837 2024-09-16T06:50:21.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-16T06:50:21.668 INFO:teuthology.orchestra.run.smithi100.stdout:94489280543 2024-09-16T06:50:21.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-16T06:50:21.689 INFO:teuthology.orchestra.run.smithi100.stdout:94489280544 2024-09-16T06:50:21.689 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-16T06:50:21.691 INFO:teuthology.orchestra.run.smithi100.stdout:98784247839 2024-09-16T06:50:21.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-16T06:50:21.700 INFO:teuthology.orchestra.run.smithi100.stdout:90194313247 2024-09-16T06:50:21.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T06:50:22.306 INFO:teuthology.orchestra.run.smithi100.stdout:94489280541 2024-09-16T06:50:22.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280542 got 94489280541 for osd.4 2024-09-16T06:50:22.398 INFO:teuthology.orchestra.run.smithi100.stdout:98784247839 2024-09-16T06:50:22.419 INFO:teuthology.orchestra.run.smithi100.stdout:98784247837 2024-09-16T06:50:22.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247840 got 98784247839 for osd.1 2024-09-16T06:50:22.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247838 got 98784247837 for osd.10 2024-09-16T06:50:22.546 INFO:teuthology.orchestra.run.smithi100.stdout:98784247838 2024-09-16T06:50:22.565 INFO:teuthology.orchestra.run.smithi100.stdout:98784247839 2024-09-16T06:50:22.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247838 got 98784247838 for osd.2 2024-09-16T06:50:22.599 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.610 INFO:teuthology.orchestra.run.smithi100.stdout:98784247837 2024-09-16T06:50:22.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247839 got 98784247839 for osd.0 2024-09-16T06:50:22.610 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.628 INFO:teuthology.orchestra.run.smithi100.stdout:94489280545 2024-09-16T06:50:22.638 INFO:teuthology.orchestra.run.smithi100.stdout:90194313247 2024-09-16T06:50:22.639 INFO:teuthology.orchestra.run.smithi100.stdout:493921239048 2024-09-16T06:50:22.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247837 got 98784247837 for osd.15 2024-09-16T06:50:22.648 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280545 got 94489280545 for osd.7 2024-09-16T06:50:22.663 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.664 INFO:teuthology.orchestra.run.smithi100.stdout:94489280543 2024-09-16T06:50:22.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 493921239048 got 493921239048 for osd.5 2024-09-16T06:50:22.691 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.694 INFO:teuthology.orchestra.run.smithi100.stdout:98784247838 2024-09-16T06:50:22.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.6 2024-09-16T06:50:22.696 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.697 INFO:teuthology.orchestra.run.smithi100.stdout:94489280544 2024-09-16T06:50:22.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.13 2024-09-16T06:50:22.701 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.715 INFO:teuthology.orchestra.run.smithi100.stdout:98784247837 2024-09-16T06:50:22.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247838 got 98784247838 for osd.11 2024-09-16T06:50:22.717 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280544 got 94489280544 for osd.12 2024-09-16T06:50:22.718 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.728 INFO:teuthology.orchestra.run.smithi100.stdout:98784247839 2024-09-16T06:50:22.734 INFO:teuthology.orchestra.run.smithi100.stdout:94489280543 2024-09-16T06:50:22.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247837 got 98784247837 for osd.9 2024-09-16T06:50:22.741 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247839 got 98784247839 for osd.14 2024-09-16T06:50:22.748 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280543 got 94489280543 for osd.8 2024-09-16T06:50:22.754 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:22.770 INFO:teuthology.orchestra.run.smithi100.stdout:90194313247 2024-09-16T06:50:22.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.3 2024-09-16T06:50:22.787 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:23.349 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T06:50:23.435 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T06:50:23.472 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-16T06:50:23.694 INFO:teuthology.orchestra.run.smithi100.stdout:94489280543 2024-09-16T06:50:23.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280542 got 94489280543 for osd.4 2024-09-16T06:50:23.712 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:23.735 INFO:teuthology.orchestra.run.smithi100.stdout:98784247840 2024-09-16T06:50:23.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247840 got 98784247840 for osd.1 2024-09-16T06:50:23.753 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:23.795 INFO:teuthology.orchestra.run.smithi100.stdout:98784247838 2024-09-16T06:50:23.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247838 got 98784247838 for osd.10 2024-09-16T06:50:23.813 DEBUG:teuthology.parallel:result is None 2024-09-16T06:50:23.813 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T06:50:23.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:24.098 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:24.099 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:24.115 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-16T06:50:23.790982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":589868,"kb_used_data":36216,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509359572,"statfs":{"total":1546188226560,"available":1545584201728,"internally_reserved":0,"allocated":37085184,"data_stored":22191520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374418"},"pg_stats":[{"pgid":"1.0","version":"28'157","reported_seq":337,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.149950+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:50:20.149950+0000","last_peered":"2024-09-16T06:50:20.149950+0000","last_clean":"2024-09-16T06:50:20.149950+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:20.149950+0000","last_undegraded":"2024-09-16T06:50:20.149950+0000","last_fullsized":"2024-09-16T06:50:20.149950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445250+0000","last_change":"2024-09-16T06:49:36.234569+0000","last_active":"2024-09-16T06:50:20.445250+0000","last_peered":"2024-09-16T06:50:20.445250+0000","last_clean":"2024-09-16T06:50:20.445250+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:20.445250+0000","last_undegraded":"2024-09-16T06:50:20.445250+0000","last_fullsized":"2024-09-16T06:50:20.445250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-16T06:49:36.234276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:36.234276+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:44.146105+0000","scrub_duration":0.0038674949999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457180+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:50:19.457180+0000","last_peered":"2024-09-16T06:50:19.457180+0000","last_clean":"2024-09-16T06:50:19.457180+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:19.457180+0000","last_undegraded":"2024-09-16T06:50:19.457180+0000","last_fullsized":"2024-09-16T06:50:19.457180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445205+0000","last_change":"2024-09-16T06:49:39.164006+0000","last_active":"2024-09-16T06:50:20.445205+0000","last_peered":"2024-09-16T06:50:20.445205+0000","last_clean":"2024-09-16T06:50:20.445205+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:20.445205+0000","last_undegraded":"2024-09-16T06:50:20.445205+0000","last_fullsized":"2024-09-16T06:50:20.445205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.163830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.163830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:50.850409+0000","scrub_duration":0.0052168600000000002,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":145,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457220+0000","last_change":"2024-09-16T06:49:39.238192+0000","last_active":"2024-09-16T06:50:19.457220+0000","last_peered":"2024-09-16T06:50:19.457220+0000","last_clean":"2024-09-16T06:50:19.457220+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:19.457220+0000","last_undegraded":"2024-09-16T06:50:19.457220+0000","last_fullsized":"2024-09-16T06:50:19.457220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-16T06:49:39.237948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.237948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:04.804819+0000","scrub_duration":0.0076161809999999996,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.571699+0000","last_change":"2024-09-16T06:49:39.555011+0000","last_active":"2024-09-16T06:50:20.571699+0000","last_peered":"2024-09-16T06:50:20.571699+0000","last_clean":"2024-09-16T06:50:20.571699+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:20.571699+0000","last_undegraded":"2024-09-16T06:50:20.571699+0000","last_fullsized":"2024-09-16T06:50:20.571699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.554796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.554796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:59.007563+0000","scrub_duration":0.0028951150000000002,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.440952+0000","last_change":"2024-09-16T06:49:50.812998+0000","last_active":"2024-09-16T06:50:19.440952+0000","last_peered":"2024-09-16T06:50:19.440952+0000","last_clean":"2024-09-16T06:50:19.440952+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:19.440952+0000","last_undegraded":"2024-09-16T06:50:19.440952+0000","last_fullsized":"2024-09-16T06:50:19.440952+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:50.812756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:50.812756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:20.298370+0000","scrub_duration":0.0062089459999999999,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.596035+0000","last_change":"2024-09-16T06:49:37.683493+0000","last_active":"2024-09-16T06:50:21.596035+0000","last_peered":"2024-09-16T06:50:21.596035+0000","last_clean":"2024-09-16T06:50:21.596035+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:21.596035+0000","last_undegraded":"2024-09-16T06:50:21.596035+0000","last_fullsized":"2024-09-16T06:50:21.596035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:37.683367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:37.683367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:54.992488+0000","scrub_duration":0.0022252779999999998,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.583068+0000","last_change":"2024-09-16T06:50:20.056944+0000","last_active":"2024-09-16T06:50:21.583068+0000","last_peered":"2024-09-16T06:50:21.583068+0000","last_clean":"2024-09-16T06:50:21.583068+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:21.583068+0000","last_undegraded":"2024-09-16T06:50:21.583068+0000","last_fullsized":"2024-09-16T06:50:21.583068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:20.056663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:50:20.056663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:34.570002+0000","scrub_duration":0.0034361130000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247838,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":0.75600000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":1.264},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.3109999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.96599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.343},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":1.4910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.54},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":1.456},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.399}]}]},{"osd":14,"up_from":23,"seq":98784247840,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51376,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320464,"statfs":{"total":96636764160,"available":96584155136,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.343}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.248},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":0.995}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.92100000000000004}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.89800000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.397},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.5469999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":577.17399999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.512},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5920000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.728},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.772}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.615},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.671}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.661}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":35.021999999999998},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.4059999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2236416,"data_stored":1310041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":1.1060000000000001},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":622.62400000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":1.02},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.282}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.202},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.1679999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.2689999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.421},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.3620000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3169999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33296,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.417},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.97699999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.0860000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.2070000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.486}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.349},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.385}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":1.466},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.258}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.4430000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247840,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2781184,"data_stored":1848560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.077},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":1.036},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":251.89099999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.107},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":0.91500000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.81200000000000006}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4060000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":1.0129999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.99299999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.94699999999999995}]}]},{"osd":4,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":107.98999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.7030000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.54200000000000004}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":1.577},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":1.105}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.7190000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.238}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":0.69199999999999995}]}]},{"osd":5,"up_from":115,"seq":493921239049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30836,"kb_used_data":2212,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94341004,"statfs":{"total":96636764160,"available":96605188096,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2764800,"data_stored":1838950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":0.623},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.753}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.615}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.587}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.302},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.9430000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.4350000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.538}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.53}]}]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82699999999999996},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":1.1639999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.754}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.026}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.282}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.1910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.379}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.90000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":1.171},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.226}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":0.94999999999999996}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0429999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.329}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":1.196},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.246},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.2969999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.369},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":1.2749999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":1.01},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.76200000000000001},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":1.2849999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.2569999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":0.878},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.179},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":0.998}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":1.3260000000000001},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":1.2170000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.4470000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.486},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3620000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247841,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37424,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.296}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.089}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.681},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.3620000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.333},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.411}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.474}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.633},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.544}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.758},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.7}]}]},{"osd":2,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33312,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.105},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":0.83799999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.1319999999999999},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.077},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":1.1739999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.397}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.266}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":1.454},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":1.371}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.4299999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":9,"up_from":23,"seq":98784247838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1329303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.085}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.115}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.159},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.212}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":1.236}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.3520000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.256}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":1.2849999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.407},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.4339999999999999}]}]},{"osd":10,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.083},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.401}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.157}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5309999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.431}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.897}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.333},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.76}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":1.9490000000000001},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":1.6160000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":2.105}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:24.119 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:24.416 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:24.417 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:24.432 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-16T06:50:23.790982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":589868,"kb_used_data":36216,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509359572,"statfs":{"total":1546188226560,"available":1545584201728,"internally_reserved":0,"allocated":37085184,"data_stored":22191520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374418"},"pg_stats":[{"pgid":"1.0","version":"28'157","reported_seq":337,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.149950+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:50:20.149950+0000","last_peered":"2024-09-16T06:50:20.149950+0000","last_clean":"2024-09-16T06:50:20.149950+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:20.149950+0000","last_undegraded":"2024-09-16T06:50:20.149950+0000","last_fullsized":"2024-09-16T06:50:20.149950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445250+0000","last_change":"2024-09-16T06:49:36.234569+0000","last_active":"2024-09-16T06:50:20.445250+0000","last_peered":"2024-09-16T06:50:20.445250+0000","last_clean":"2024-09-16T06:50:20.445250+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:20.445250+0000","last_undegraded":"2024-09-16T06:50:20.445250+0000","last_fullsized":"2024-09-16T06:50:20.445250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-16T06:49:36.234276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:36.234276+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:44.146105+0000","scrub_duration":0.0038674949999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457180+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:50:19.457180+0000","last_peered":"2024-09-16T06:50:19.457180+0000","last_clean":"2024-09-16T06:50:19.457180+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:19.457180+0000","last_undegraded":"2024-09-16T06:50:19.457180+0000","last_fullsized":"2024-09-16T06:50:19.457180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445205+0000","last_change":"2024-09-16T06:49:39.164006+0000","last_active":"2024-09-16T06:50:20.445205+0000","last_peered":"2024-09-16T06:50:20.445205+0000","last_clean":"2024-09-16T06:50:20.445205+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:20.445205+0000","last_undegraded":"2024-09-16T06:50:20.445205+0000","last_fullsized":"2024-09-16T06:50:20.445205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.163830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.163830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:50.850409+0000","scrub_duration":0.0052168600000000002,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":145,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457220+0000","last_change":"2024-09-16T06:49:39.238192+0000","last_active":"2024-09-16T06:50:19.457220+0000","last_peered":"2024-09-16T06:50:19.457220+0000","last_clean":"2024-09-16T06:50:19.457220+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:19.457220+0000","last_undegraded":"2024-09-16T06:50:19.457220+0000","last_fullsized":"2024-09-16T06:50:19.457220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-16T06:49:39.237948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.237948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:04.804819+0000","scrub_duration":0.0076161809999999996,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.571699+0000","last_change":"2024-09-16T06:49:39.555011+0000","last_active":"2024-09-16T06:50:20.571699+0000","last_peered":"2024-09-16T06:50:20.571699+0000","last_clean":"2024-09-16T06:50:20.571699+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:20.571699+0000","last_undegraded":"2024-09-16T06:50:20.571699+0000","last_fullsized":"2024-09-16T06:50:20.571699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.554796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.554796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:59.007563+0000","scrub_duration":0.0028951150000000002,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.440952+0000","last_change":"2024-09-16T06:49:50.812998+0000","last_active":"2024-09-16T06:50:19.440952+0000","last_peered":"2024-09-16T06:50:19.440952+0000","last_clean":"2024-09-16T06:50:19.440952+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:19.440952+0000","last_undegraded":"2024-09-16T06:50:19.440952+0000","last_fullsized":"2024-09-16T06:50:19.440952+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:50.812756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:50.812756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:20.298370+0000","scrub_duration":0.0062089459999999999,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.596035+0000","last_change":"2024-09-16T06:49:37.683493+0000","last_active":"2024-09-16T06:50:21.596035+0000","last_peered":"2024-09-16T06:50:21.596035+0000","last_clean":"2024-09-16T06:50:21.596035+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:21.596035+0000","last_undegraded":"2024-09-16T06:50:21.596035+0000","last_fullsized":"2024-09-16T06:50:21.596035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:37.683367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:37.683367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:54.992488+0000","scrub_duration":0.0022252779999999998,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.583068+0000","last_change":"2024-09-16T06:50:20.056944+0000","last_active":"2024-09-16T06:50:21.583068+0000","last_peered":"2024-09-16T06:50:21.583068+0000","last_clean":"2024-09-16T06:50:21.583068+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:21.583068+0000","last_undegraded":"2024-09-16T06:50:21.583068+0000","last_fullsized":"2024-09-16T06:50:21.583068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:20.056663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:50:20.056663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:34.570002+0000","scrub_duration":0.0034361130000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247838,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":0.75600000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":1.264},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.3109999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.96599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.343},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":1.4910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.54},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":1.456},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.399}]}]},{"osd":14,"up_from":23,"seq":98784247840,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51376,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320464,"statfs":{"total":96636764160,"available":96584155136,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.343}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.248},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":0.995}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.92100000000000004}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.89800000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.397},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.5469999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":577.17399999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.512},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5920000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.728},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.772}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.615},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.671}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.661}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":35.021999999999998},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.4059999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2236416,"data_stored":1310041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":1.1060000000000001},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":622.62400000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":1.02},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.282}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.202},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.1679999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.2689999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.421},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.3620000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3169999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33296,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.417},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.97699999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.0860000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.2070000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.486}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.349},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.385}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":1.466},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.258}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.4430000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247840,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2781184,"data_stored":1848560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.077},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":1.036},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":251.89099999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.107},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":0.91500000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.81200000000000006}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4060000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":1.0129999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.99299999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.94699999999999995}]}]},{"osd":4,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":107.98999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.7030000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.54200000000000004}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":1.577},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":1.105}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.7190000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.238}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":0.69199999999999995}]}]},{"osd":5,"up_from":115,"seq":493921239049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30836,"kb_used_data":2212,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94341004,"statfs":{"total":96636764160,"available":96605188096,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2764800,"data_stored":1838950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":0.623},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.753}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.615}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.587}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.302},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.9430000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.4350000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.538}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.53}]}]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82699999999999996},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":1.1639999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.754}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.026}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.282}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.1910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.379}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.90000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":1.171},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.226}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":0.94999999999999996}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0429999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.329}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":1.196},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.246},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.2969999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.369},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":1.2749999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":1.01},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.76200000000000001},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":1.2849999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.2569999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":0.878},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.179},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":0.998}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":1.3260000000000001},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":1.2170000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.4470000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.486},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3620000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247841,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37424,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.296}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.089}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.681},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.3620000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.333},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.411}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.474}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.633},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.544}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.758},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.7}]}]},{"osd":2,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33312,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.105},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":0.83799999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.1319999999999999},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.077},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":1.1739999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.397}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.266}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":1.454},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":1.371}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.4299999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":9,"up_from":23,"seq":98784247838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1329303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.085}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.115}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.159},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.212}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":1.236}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.3520000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.256}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":1.2849999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.407},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.4339999999999999}]}]},{"osd":10,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.083},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.401}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.157}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5309999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.431}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.897}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.333},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.76}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":1.9490000000000001},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":1.6160000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":2.105}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:24.436 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T06:50:24.436 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T06:50:24.739 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T06:50:24.749 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T06:50:24.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:25.042 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:25.042 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:25.058 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-16T06:50:23.790982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":589868,"kb_used_data":36216,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509359572,"statfs":{"total":1546188226560,"available":1545584201728,"internally_reserved":0,"allocated":37085184,"data_stored":22191520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374418"},"pg_stats":[{"pgid":"1.0","version":"28'157","reported_seq":337,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.149950+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:50:20.149950+0000","last_peered":"2024-09-16T06:50:20.149950+0000","last_clean":"2024-09-16T06:50:20.149950+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:20.149950+0000","last_undegraded":"2024-09-16T06:50:20.149950+0000","last_fullsized":"2024-09-16T06:50:20.149950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445250+0000","last_change":"2024-09-16T06:49:36.234569+0000","last_active":"2024-09-16T06:50:20.445250+0000","last_peered":"2024-09-16T06:50:20.445250+0000","last_clean":"2024-09-16T06:50:20.445250+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:20.445250+0000","last_undegraded":"2024-09-16T06:50:20.445250+0000","last_fullsized":"2024-09-16T06:50:20.445250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-16T06:49:36.234276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:36.234276+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:44.146105+0000","scrub_duration":0.0038674949999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457180+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:50:19.457180+0000","last_peered":"2024-09-16T06:50:19.457180+0000","last_clean":"2024-09-16T06:50:19.457180+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:19.457180+0000","last_undegraded":"2024-09-16T06:50:19.457180+0000","last_fullsized":"2024-09-16T06:50:19.457180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445205+0000","last_change":"2024-09-16T06:49:39.164006+0000","last_active":"2024-09-16T06:50:20.445205+0000","last_peered":"2024-09-16T06:50:20.445205+0000","last_clean":"2024-09-16T06:50:20.445205+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:20.445205+0000","last_undegraded":"2024-09-16T06:50:20.445205+0000","last_fullsized":"2024-09-16T06:50:20.445205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.163830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.163830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:50.850409+0000","scrub_duration":0.0052168600000000002,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":145,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457220+0000","last_change":"2024-09-16T06:49:39.238192+0000","last_active":"2024-09-16T06:50:19.457220+0000","last_peered":"2024-09-16T06:50:19.457220+0000","last_clean":"2024-09-16T06:50:19.457220+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:19.457220+0000","last_undegraded":"2024-09-16T06:50:19.457220+0000","last_fullsized":"2024-09-16T06:50:19.457220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-16T06:49:39.237948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.237948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:04.804819+0000","scrub_duration":0.0076161809999999996,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.571699+0000","last_change":"2024-09-16T06:49:39.555011+0000","last_active":"2024-09-16T06:50:20.571699+0000","last_peered":"2024-09-16T06:50:20.571699+0000","last_clean":"2024-09-16T06:50:20.571699+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:20.571699+0000","last_undegraded":"2024-09-16T06:50:20.571699+0000","last_fullsized":"2024-09-16T06:50:20.571699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.554796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.554796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:59.007563+0000","scrub_duration":0.0028951150000000002,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.440952+0000","last_change":"2024-09-16T06:49:50.812998+0000","last_active":"2024-09-16T06:50:19.440952+0000","last_peered":"2024-09-16T06:50:19.440952+0000","last_clean":"2024-09-16T06:50:19.440952+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:19.440952+0000","last_undegraded":"2024-09-16T06:50:19.440952+0000","last_fullsized":"2024-09-16T06:50:19.440952+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:50.812756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:50.812756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:20.298370+0000","scrub_duration":0.0062089459999999999,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.596035+0000","last_change":"2024-09-16T06:49:37.683493+0000","last_active":"2024-09-16T06:50:21.596035+0000","last_peered":"2024-09-16T06:50:21.596035+0000","last_clean":"2024-09-16T06:50:21.596035+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:21.596035+0000","last_undegraded":"2024-09-16T06:50:21.596035+0000","last_fullsized":"2024-09-16T06:50:21.596035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:37.683367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:37.683367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:54.992488+0000","scrub_duration":0.0022252779999999998,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.583068+0000","last_change":"2024-09-16T06:50:20.056944+0000","last_active":"2024-09-16T06:50:21.583068+0000","last_peered":"2024-09-16T06:50:21.583068+0000","last_clean":"2024-09-16T06:50:21.583068+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:21.583068+0000","last_undegraded":"2024-09-16T06:50:21.583068+0000","last_fullsized":"2024-09-16T06:50:21.583068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:20.056663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:50:20.056663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:34.570002+0000","scrub_duration":0.0034361130000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247838,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":0.75600000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":1.264},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.3109999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.96599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.343},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":1.4910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.54},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":1.456},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.399}]}]},{"osd":14,"up_from":23,"seq":98784247840,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51376,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320464,"statfs":{"total":96636764160,"available":96584155136,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.343}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.248},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":0.995}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.92100000000000004}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.89800000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.397},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.5469999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":577.17399999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.512},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5920000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.728},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.772}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.615},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.671}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.661}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":35.021999999999998},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.4059999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2236416,"data_stored":1310041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":1.1060000000000001},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":622.62400000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":1.02},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.282}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.202},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.1679999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.2689999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.421},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.3620000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3169999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33296,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.417},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.97699999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.0860000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.2070000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.486}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.349},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.385}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":1.466},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.258}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.4430000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247840,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2781184,"data_stored":1848560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.077},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":1.036},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":251.89099999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.107},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":0.91500000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.81200000000000006}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4060000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":1.0129999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.99299999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.94699999999999995}]}]},{"osd":4,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":107.98999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.7030000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.54200000000000004}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":1.577},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":1.105}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.7190000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.238}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":0.69199999999999995}]}]},{"osd":5,"up_from":115,"seq":493921239049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30836,"kb_used_data":2212,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94341004,"statfs":{"total":96636764160,"available":96605188096,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2764800,"data_stored":1838950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":0.623},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.753}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.615}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.587}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.302},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.9430000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.4350000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.538}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.53}]}]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82699999999999996},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":1.1639999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.754}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.026}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.282}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.1910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.379}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.90000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":1.171},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.226}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":0.94999999999999996}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0429999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.329}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":1.196},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.246},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.2969999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.369},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":1.2749999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":1.01},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.76200000000000001},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":1.2849999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.2569999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":0.878},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.179},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":0.998}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":1.3260000000000001},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":1.2170000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.4470000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.486},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3620000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247841,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37424,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.296}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.089}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.681},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.3620000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.333},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.411}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.474}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.633},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.544}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.758},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.7}]}]},{"osd":2,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33312,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.105},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":0.83799999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.1319999999999999},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.077},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":1.1739999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.397}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.266}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":1.454},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":1.371}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.4299999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":9,"up_from":23,"seq":98784247838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1329303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.085}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.115}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.159},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.212}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":1.236}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.3520000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.256}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":1.2849999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.407},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.4339999999999999}]}]},{"osd":10,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.083},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.401}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.157}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5309999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.431}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.897}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.333},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.76}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":1.9490000000000001},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":1.6160000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":2.105}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:25.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:25.351 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:25.351 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:25.367 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-16T06:50:23.790982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":589868,"kb_used_data":36216,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509359572,"statfs":{"total":1546188226560,"available":1545584201728,"internally_reserved":0,"allocated":37085184,"data_stored":22191520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374418"},"pg_stats":[{"pgid":"1.0","version":"28'157","reported_seq":337,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.149950+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:50:20.149950+0000","last_peered":"2024-09-16T06:50:20.149950+0000","last_clean":"2024-09-16T06:50:20.149950+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:20.149950+0000","last_undegraded":"2024-09-16T06:50:20.149950+0000","last_fullsized":"2024-09-16T06:50:20.149950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445250+0000","last_change":"2024-09-16T06:49:36.234569+0000","last_active":"2024-09-16T06:50:20.445250+0000","last_peered":"2024-09-16T06:50:20.445250+0000","last_clean":"2024-09-16T06:50:20.445250+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:20.445250+0000","last_undegraded":"2024-09-16T06:50:20.445250+0000","last_fullsized":"2024-09-16T06:50:20.445250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-16T06:49:36.234276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:36.234276+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:44.146105+0000","scrub_duration":0.0038674949999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457180+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:50:19.457180+0000","last_peered":"2024-09-16T06:50:19.457180+0000","last_clean":"2024-09-16T06:50:19.457180+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:19.457180+0000","last_undegraded":"2024-09-16T06:50:19.457180+0000","last_fullsized":"2024-09-16T06:50:19.457180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445205+0000","last_change":"2024-09-16T06:49:39.164006+0000","last_active":"2024-09-16T06:50:20.445205+0000","last_peered":"2024-09-16T06:50:20.445205+0000","last_clean":"2024-09-16T06:50:20.445205+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:20.445205+0000","last_undegraded":"2024-09-16T06:50:20.445205+0000","last_fullsized":"2024-09-16T06:50:20.445205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.163830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.163830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:50.850409+0000","scrub_duration":0.0052168600000000002,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":145,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457220+0000","last_change":"2024-09-16T06:49:39.238192+0000","last_active":"2024-09-16T06:50:19.457220+0000","last_peered":"2024-09-16T06:50:19.457220+0000","last_clean":"2024-09-16T06:50:19.457220+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:19.457220+0000","last_undegraded":"2024-09-16T06:50:19.457220+0000","last_fullsized":"2024-09-16T06:50:19.457220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-16T06:49:39.237948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.237948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:04.804819+0000","scrub_duration":0.0076161809999999996,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.571699+0000","last_change":"2024-09-16T06:49:39.555011+0000","last_active":"2024-09-16T06:50:20.571699+0000","last_peered":"2024-09-16T06:50:20.571699+0000","last_clean":"2024-09-16T06:50:20.571699+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:20.571699+0000","last_undegraded":"2024-09-16T06:50:20.571699+0000","last_fullsized":"2024-09-16T06:50:20.571699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.554796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.554796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:59.007563+0000","scrub_duration":0.0028951150000000002,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.440952+0000","last_change":"2024-09-16T06:49:50.812998+0000","last_active":"2024-09-16T06:50:19.440952+0000","last_peered":"2024-09-16T06:50:19.440952+0000","last_clean":"2024-09-16T06:50:19.440952+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:19.440952+0000","last_undegraded":"2024-09-16T06:50:19.440952+0000","last_fullsized":"2024-09-16T06:50:19.440952+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:50.812756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:50.812756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:20.298370+0000","scrub_duration":0.0062089459999999999,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.596035+0000","last_change":"2024-09-16T06:49:37.683493+0000","last_active":"2024-09-16T06:50:21.596035+0000","last_peered":"2024-09-16T06:50:21.596035+0000","last_clean":"2024-09-16T06:50:21.596035+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:21.596035+0000","last_undegraded":"2024-09-16T06:50:21.596035+0000","last_fullsized":"2024-09-16T06:50:21.596035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:37.683367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:37.683367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:54.992488+0000","scrub_duration":0.0022252779999999998,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.583068+0000","last_change":"2024-09-16T06:50:20.056944+0000","last_active":"2024-09-16T06:50:21.583068+0000","last_peered":"2024-09-16T06:50:21.583068+0000","last_clean":"2024-09-16T06:50:21.583068+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:21.583068+0000","last_undegraded":"2024-09-16T06:50:21.583068+0000","last_fullsized":"2024-09-16T06:50:21.583068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:20.056663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:50:20.056663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:34.570002+0000","scrub_duration":0.0034361130000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247838,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":0.75600000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":1.264},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.3109999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.96599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.343},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":1.4910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.54},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":1.456},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.399}]}]},{"osd":14,"up_from":23,"seq":98784247840,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51376,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320464,"statfs":{"total":96636764160,"available":96584155136,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.343}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.248},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":0.995}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.92100000000000004}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.89800000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.397},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.5469999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":577.17399999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.512},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5920000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.728},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.772}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.615},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.671}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.661}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":35.021999999999998},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.4059999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2236416,"data_stored":1310041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":1.1060000000000001},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":622.62400000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":1.02},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.282}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.202},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.1679999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.2689999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.421},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.3620000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3169999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33296,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.417},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.97699999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.0860000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.2070000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.486}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.349},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.385}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":1.466},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.258}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.4430000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247840,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2781184,"data_stored":1848560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.077},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":1.036},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":251.89099999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.107},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":0.91500000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.81200000000000006}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4060000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":1.0129999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.99299999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.94699999999999995}]}]},{"osd":4,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":107.98999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.7030000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.54200000000000004}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":1.577},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":1.105}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.7190000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.238}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":0.69199999999999995}]}]},{"osd":5,"up_from":115,"seq":493921239049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30836,"kb_used_data":2212,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94341004,"statfs":{"total":96636764160,"available":96605188096,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2764800,"data_stored":1838950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":0.623},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.753}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.615}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.587}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.302},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.9430000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.4350000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.538}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.53}]}]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82699999999999996},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":1.1639999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.754}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.026}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.282}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.1910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.379}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.90000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":1.171},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.226}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":0.94999999999999996}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0429999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.329}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":1.196},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.246},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.2969999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.369},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":1.2749999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":1.01},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.76200000000000001},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":1.2849999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.2569999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":0.878},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.179},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":0.998}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":1.3260000000000001},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":1.2170000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.4470000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.486},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3620000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247841,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37424,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.296}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.089}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.681},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.3620000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.333},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.411}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.474}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.633},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.544}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.758},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.7}]}]},{"osd":2,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33312,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.105},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":0.83799999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.1319999999999999},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.077},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":1.1739999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.397}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.266}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":1.454},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":1.371}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.4299999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":9,"up_from":23,"seq":98784247838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1329303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.085}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.115}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.159},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.212}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":1.236}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.3520000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.256}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":1.2849999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.407},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.4339999999999999}]}]},{"osd":10,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.083},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.401}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.157}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5309999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.431}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.897}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.333},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.76}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":1.9490000000000001},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":1.6160000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":2.105}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:25.371 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T06:50:25.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:25.656 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:25.657 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:25.672 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-16T06:50:23.790982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":589868,"kb_used_data":36216,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509359572,"statfs":{"total":1546188226560,"available":1545584201728,"internally_reserved":0,"allocated":37085184,"data_stored":22191520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374418"},"pg_stats":[{"pgid":"1.0","version":"28'157","reported_seq":337,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.149950+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:50:20.149950+0000","last_peered":"2024-09-16T06:50:20.149950+0000","last_clean":"2024-09-16T06:50:20.149950+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:20.149950+0000","last_undegraded":"2024-09-16T06:50:20.149950+0000","last_fullsized":"2024-09-16T06:50:20.149950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445250+0000","last_change":"2024-09-16T06:49:36.234569+0000","last_active":"2024-09-16T06:50:20.445250+0000","last_peered":"2024-09-16T06:50:20.445250+0000","last_clean":"2024-09-16T06:50:20.445250+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:20.445250+0000","last_undegraded":"2024-09-16T06:50:20.445250+0000","last_fullsized":"2024-09-16T06:50:20.445250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-16T06:49:36.234276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:36.234276+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:44.146105+0000","scrub_duration":0.0038674949999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457180+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:50:19.457180+0000","last_peered":"2024-09-16T06:50:19.457180+0000","last_clean":"2024-09-16T06:50:19.457180+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:19.457180+0000","last_undegraded":"2024-09-16T06:50:19.457180+0000","last_fullsized":"2024-09-16T06:50:19.457180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.445205+0000","last_change":"2024-09-16T06:49:39.164006+0000","last_active":"2024-09-16T06:50:20.445205+0000","last_peered":"2024-09-16T06:50:20.445205+0000","last_clean":"2024-09-16T06:50:20.445205+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:20.445205+0000","last_undegraded":"2024-09-16T06:50:20.445205+0000","last_fullsized":"2024-09-16T06:50:20.445205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.163830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.163830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:50.850409+0000","scrub_duration":0.0052168600000000002,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":145,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.457220+0000","last_change":"2024-09-16T06:49:39.238192+0000","last_active":"2024-09-16T06:50:19.457220+0000","last_peered":"2024-09-16T06:50:19.457220+0000","last_clean":"2024-09-16T06:50:19.457220+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:19.457220+0000","last_undegraded":"2024-09-16T06:50:19.457220+0000","last_fullsized":"2024-09-16T06:50:19.457220+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-16T06:49:39.237948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.237948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:04.804819+0000","scrub_duration":0.0076161809999999996,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:20.571699+0000","last_change":"2024-09-16T06:49:39.555011+0000","last_active":"2024-09-16T06:50:20.571699+0000","last_peered":"2024-09-16T06:50:20.571699+0000","last_clean":"2024-09-16T06:50:20.571699+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:20.571699+0000","last_undegraded":"2024-09-16T06:50:20.571699+0000","last_fullsized":"2024-09-16T06:50:20.571699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.554796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.554796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:59.007563+0000","scrub_duration":0.0028951150000000002,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T06:50:19.440952+0000","last_change":"2024-09-16T06:49:50.812998+0000","last_active":"2024-09-16T06:50:19.440952+0000","last_peered":"2024-09-16T06:50:19.440952+0000","last_clean":"2024-09-16T06:50:19.440952+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:19.440952+0000","last_undegraded":"2024-09-16T06:50:19.440952+0000","last_fullsized":"2024-09-16T06:50:19.440952+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:50.812756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:50.812756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:20.298370+0000","scrub_duration":0.0062089459999999999,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.596035+0000","last_change":"2024-09-16T06:49:37.683493+0000","last_active":"2024-09-16T06:50:21.596035+0000","last_peered":"2024-09-16T06:50:21.596035+0000","last_clean":"2024-09-16T06:50:21.596035+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:21.596035+0000","last_undegraded":"2024-09-16T06:50:21.596035+0000","last_fullsized":"2024-09-16T06:50:21.596035+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:37.683367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:37.683367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:54.992488+0000","scrub_duration":0.0022252779999999998,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.583068+0000","last_change":"2024-09-16T06:50:20.056944+0000","last_active":"2024-09-16T06:50:21.583068+0000","last_peered":"2024-09-16T06:50:21.583068+0000","last_clean":"2024-09-16T06:50:21.583068+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:21.583068+0000","last_undegraded":"2024-09-16T06:50:21.583068+0000","last_fullsized":"2024-09-16T06:50:21.583068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:20.056663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:50:20.056663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:34.570002+0000","scrub_duration":0.0034361130000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247838,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":0.75600000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":1.264},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.3109999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":60.313000000000002,"5min":60.313000000000002,"15min":60.313000000000002},"last":0.96599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.343},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":1.4910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.54},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":1.456},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.399}]}]},{"osd":14,"up_from":23,"seq":98784247840,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51376,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320464,"statfs":{"total":96636764160,"available":96584155136,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.343}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.248},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":0.995}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.92100000000000004}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.89800000000000002}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.397},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.5469999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":577.17399999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.512},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:49:23 2024","interfaces":[{"interface":"back","average":{"1min":41.350999999999999,"5min":41.350999999999999,"15min":41.350999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":722.72199999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5920000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.728},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.772}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.615},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.671}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.661}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":35.021999999999998},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.4059999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2236416,"data_stored":1310041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":1.1060000000000001},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":622.62400000000002}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":1.02},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.282}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":1.202},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.1679999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.2689999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.421},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.3620000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3169999999999999}]}]},{"osd":13,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33296,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":0.89100000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.417},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.97699999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":4.5119999999999996,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.0860000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.171},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.2070000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":4.7160000000000002,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":17.895,"5min":17.895,"15min":17.895},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":210.85599999999999,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.486}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.349},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.385}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":1.466},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.258}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.4430000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247840,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2781184,"data_stored":1848560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":1.077},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":1.036},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":251.89099999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.125}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.107},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":0.91500000000000004}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.81200000000000006}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4060000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":1.0129999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.99299999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.94699999999999995}]}]},{"osd":4,"up_from":22,"seq":94489280543,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":107.98999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.7030000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.54200000000000004}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":1.0169999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":1.577},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":1.105}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.7190000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.238}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":0.69199999999999995}]}]},{"osd":5,"up_from":115,"seq":493921239049,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30836,"kb_used_data":2212,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94341004,"statfs":{"total":96636764160,"available":96605188096,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2350000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":6,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2764800,"data_stored":1838950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":0.623},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.753}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.0389999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.615}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.587}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.302},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.9430000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.4350000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.538}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":15,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.53}]}]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2232320,"data_stored":1310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82699999999999996},{"interface":"front","average":{"1min":12.967000000000001,"5min":12.967000000000001,"15min":12.967000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":305.589,"5min":305.589,"15min":305.589},"last":1.1639999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.754}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2490000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.026}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.282}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.133}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.1910000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.379}]}]},{"osd":8,"up_from":22,"seq":94489280543,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.90000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":1.171},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":1.143}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.226}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":0.94999999999999996}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0429999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":4.4960000000000004,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.329}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":1.196},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.095}]},{"osd":14,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.246},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":5.5640000000000001,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.2969999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.369},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":1.2749999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":1.01},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.85699999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.76200000000000001},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":1.2849999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.2569999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":0.878},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.68899999999999995}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.179},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":0.998}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":1.3260000000000001},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":1.2170000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.4470000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":33.183999999999997,"5min":33.183999999999997,"15min":33.183999999999997},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":575.04499999999996,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.486},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3620000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247841,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37424,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334416,"statfs":{"total":96636764160,"available":96598441984,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.296}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.089}]},{"osd":10,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.681},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.3620000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.333},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.411}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.474}]},{"osd":13,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.633},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.544}]},{"osd":14,"last update":"Mon Sep 16 06:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.758},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.7}]}]},{"osd":2,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33312,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2248704,"data_stored":1319632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.105},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":0.83799999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.1319999999999999},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.077},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":1.1739999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.397}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.266}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":1.454},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":1.371}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.4299999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":9,"up_from":23,"seq":98784247838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2269184,"data_stored":1329303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":0.68899999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.085}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.115}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.159},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.212}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":1.236}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.3520000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.256}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":1.2849999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.407},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.4339999999999999}]}]},{"osd":10,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51380,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2265088,"data_stored":1329284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.083},{"interface":"front","average":{"1min":109.21599999999999,"5min":109.21599999999999,"15min":109.21599999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":981.82899999999995,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.401}]},{"osd":4,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.157}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.5309999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.431}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.897}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.333},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.76}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":1.9490000000000001},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":1.6160000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":2.105}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:25.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T06:50:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:25.959 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":148,"fsid":"b2976256-637c-4e28-94f6-99f20017bc5e","created":"2024-09-16T06:47:53.157733+0000","modified":"2024-09-16T06:50:21.445415+0000","last_up_change":"2024-09-16T06:49:48.103739+0000","last_in_change":"2024-09-16T06:49:04.862161+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T06:48:05.838641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T06:48:08.489111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":28,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54494294-f25f-41f2-bff7-106492665e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6809","nonce":1028275949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6811","nonce":1028275949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6815","nonce":1028275949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1028275949},{"type":"v1","addr":"172.21.15.100:6813","nonce":1028275949}]},"public_addr":"172.21.15.100:6809/1028275949","cluster_addr":"172.21.15.100:6811/1028275949","heartbeat_back_addr":"172.21.15.100:6815/1028275949","heartbeat_front_addr":"172.21.15.100:6813/1028275949","state":["exists","up"]},{"osd":1,"uuid":"a265e752-1cd6-497e-bcb9-c56591a5ce30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6809","nonce":930331367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6811","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6813","nonce":930331367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6819","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6821","nonce":930331367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6815","nonce":930331367},{"type":"v1","addr":"172.21.15.107:6817","nonce":930331367}]},"public_addr":"172.21.15.107:6809/930331367","cluster_addr":"172.21.15.107:6813/930331367","heartbeat_back_addr":"172.21.15.107:6821/930331367","heartbeat_front_addr":"172.21.15.107:6817/930331367","state":["exists","up"]},{"osd":2,"uuid":"dada34c6-4dc6-4f9f-bc31-85983fd2d7fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6801","nonce":4231506893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6803","nonce":4231506893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6807","nonce":4231506893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4231506893},{"type":"v1","addr":"172.21.15.115:6805","nonce":4231506893}]},"public_addr":"172.21.15.115:6801/4231506893","cluster_addr":"172.21.15.115:6803/4231506893","heartbeat_back_addr":"172.21.15.115:6807/4231506893","heartbeat_front_addr":"172.21.15.115:6805/4231506893","state":["exists","up"]},{"osd":3,"uuid":"6c0fc051-a4a0-4ce5-989a-668e75bbe3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6803","nonce":1995864012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6805","nonce":1995864012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6809","nonce":1995864012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1995864012},{"type":"v1","addr":"172.21.15.117:6807","nonce":1995864012}]},"public_addr":"172.21.15.117:6803/1995864012","cluster_addr":"172.21.15.117:6805/1995864012","heartbeat_back_addr":"172.21.15.117:6809/1995864012","heartbeat_front_addr":"172.21.15.117:6807/1995864012","state":["exists","up"]},{"osd":4,"uuid":"36a46387-b859-4937-bbd0-5dd708630c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6801","nonce":2054405220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6803","nonce":2054405220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6807","nonce":2054405220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2054405220},{"type":"v1","addr":"172.21.15.100:6805","nonce":2054405220}]},"public_addr":"172.21.15.100:6801/2054405220","cluster_addr":"172.21.15.100:6803/2054405220","heartbeat_back_addr":"172.21.15.100:6807/2054405220","heartbeat_front_addr":"172.21.15.100:6805/2054405220","state":["exists","up"]},{"osd":5,"uuid":"073f3d23-77f0-4fd3-b067-27fa81137288","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":82,"up_from":115,"up_thru":37,"down_at":83,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6812","nonce":1072021825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6816","nonce":1072021825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6823","nonce":1072021825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1072021825},{"type":"v1","addr":"172.21.15.107:6820","nonce":1072021825}]},"public_addr":"172.21.15.107:6812/1072021825","cluster_addr":"172.21.15.107:6816/1072021825","heartbeat_back_addr":"172.21.15.107:6823/1072021825","heartbeat_front_addr":"172.21.15.107:6820/1072021825","state":["exists","up"]},{"osd":6,"uuid":"8a81592f-3704-48f6-89ba-cc4997c2105e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6825","nonce":1095047955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6827","nonce":1095047955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6831","nonce":1095047955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1095047955},{"type":"v1","addr":"172.21.15.115:6829","nonce":1095047955}]},"public_addr":"172.21.15.115:6825/1095047955","cluster_addr":"172.21.15.115:6827/1095047955","heartbeat_back_addr":"172.21.15.115:6831/1095047955","heartbeat_front_addr":"172.21.15.115:6829/1095047955","state":["exists","up"]},{"osd":7,"uuid":"3f1dcc9b-e0ed-4dc5-99e7-02f7ca196f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6819","nonce":3607859516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6821","nonce":3607859516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6825","nonce":3607859516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":3607859516},{"type":"v1","addr":"172.21.15.117:6823","nonce":3607859516}]},"public_addr":"172.21.15.117:6819/3607859516","cluster_addr":"172.21.15.117:6821/3607859516","heartbeat_back_addr":"172.21.15.117:6825/3607859516","heartbeat_front_addr":"172.21.15.117:6823/3607859516","state":["exists","up"]},{"osd":8,"uuid":"7ce38131-057f-4d45-88a5-dae8bcceae3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6817","nonce":232055141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6819","nonce":232055141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6823","nonce":232055141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":232055141},{"type":"v1","addr":"172.21.15.100:6821","nonce":232055141}]},"public_addr":"172.21.15.100:6817/232055141","cluster_addr":"172.21.15.100:6819/232055141","heartbeat_back_addr":"172.21.15.100:6823/232055141","heartbeat_front_addr":"172.21.15.100:6821/232055141","state":["exists","up"]},{"osd":9,"uuid":"aa7fde6e-6563-4e37-8af2-9114d0091362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6825","nonce":1511881207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6827","nonce":1511881207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6831","nonce":1511881207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1511881207},{"type":"v1","addr":"172.21.15.107:6829","nonce":1511881207}]},"public_addr":"172.21.15.107:6825/1511881207","cluster_addr":"172.21.15.107:6827/1511881207","heartbeat_back_addr":"172.21.15.107:6831/1511881207","heartbeat_front_addr":"172.21.15.107:6829/1511881207","state":["exists","up"]},{"osd":10,"uuid":"05c5389d-8082-4be3-aa6d-138e00720f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6809","nonce":435900412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6811","nonce":435900412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6815","nonce":435900412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":435900412},{"type":"v1","addr":"172.21.15.115:6813","nonce":435900412}]},"public_addr":"172.21.15.115:6809/435900412","cluster_addr":"172.21.15.115:6811/435900412","heartbeat_back_addr":"172.21.15.115:6815/435900412","heartbeat_front_addr":"172.21.15.115:6813/435900412","state":["exists","up"]},{"osd":11,"uuid":"99552590-1d0a-420e-b0c3-357246880658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6811","nonce":3208528083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6813","nonce":3208528083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6817","nonce":3208528083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":3208528083},{"type":"v1","addr":"172.21.15.117:6815","nonce":3208528083}]},"public_addr":"172.21.15.117:6811/3208528083","cluster_addr":"172.21.15.117:6813/3208528083","heartbeat_back_addr":"172.21.15.117:6817/3208528083","heartbeat_front_addr":"172.21.15.117:6815/3208528083","state":["exists","up"]},{"osd":12,"uuid":"165fea1c-74b6-4806-a0d9-47441e161ed5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6825","nonce":3107910928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6827","nonce":3107910928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6831","nonce":3107910928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3107910928},{"type":"v1","addr":"172.21.15.100:6829","nonce":3107910928}]},"public_addr":"172.21.15.100:6825/3107910928","cluster_addr":"172.21.15.100:6827/3107910928","heartbeat_back_addr":"172.21.15.100:6831/3107910928","heartbeat_front_addr":"172.21.15.100:6829/3107910928","state":["exists","up"]},{"osd":13,"uuid":"03402d0c-e7fd-4233-b2cd-bf79cd75e9bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6801","nonce":316232366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6803","nonce":316232366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6807","nonce":316232366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":316232366},{"type":"v1","addr":"172.21.15.107:6805","nonce":316232366}]},"public_addr":"172.21.15.107:6801/316232366","cluster_addr":"172.21.15.107:6803/316232366","heartbeat_back_addr":"172.21.15.107:6807/316232366","heartbeat_front_addr":"172.21.15.107:6805/316232366","state":["exists","up"]},{"osd":14,"uuid":"7b027faf-5c9e-4aee-9cb9-9c6459b8fe39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6817","nonce":693497656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6819","nonce":693497656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6823","nonce":693497656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":693497656},{"type":"v1","addr":"172.21.15.115:6821","nonce":693497656}]},"public_addr":"172.21.15.115:6817/693497656","cluster_addr":"172.21.15.115:6819/693497656","heartbeat_back_addr":"172.21.15.115:6823/693497656","heartbeat_front_addr":"172.21.15.115:6821/693497656","state":["exists","up"]},{"osd":15,"uuid":"4877aa40-c0d8-494f-8a98-36089ed694c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6827","nonce":579912865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6829","nonce":579912865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6833","nonce":579912865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":579912865},{"type":"v1","addr":"172.21.15.117:6831","nonce":579912865}]},"public_addr":"172.21.15.117:6827/579912865","cluster_addr":"172.21.15.117:6829/579912865","heartbeat_back_addr":"172.21.15.117:6833/579912865","heartbeat_front_addr":"172.21.15.117:6831/579912865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.319627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:31.889358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.542092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:28.191679+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:17.579785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:48.020278+0000","dead_epoch":83},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:21.476717+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:11.670056+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:19.668799+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:25.819931+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:50:06.725344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:50:20.613940+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:15.898585+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:30.022637+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:22.810301+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T06:49:33.012714+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":21,"length":1},{"begin":28,"length":2},{"begin":31,"length":3},{"begin":41,"length":4},{"begin":46,"length":1},{"begin":51,"length":1},{"begin":53,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T06:50:26.977 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T06:50:26.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:27.148 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:27.149 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:27.149 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:27.165 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T06:50:27.449 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T06:50:27.466 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T06:50:27.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:27.615 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:27.615 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:27.615 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:27.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T06:50:27.920 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T06:50:27.937 INFO:tasks.ceph:Scrubbing osd.8 2024-09-16T06:50:27.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:28.081 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:28.081 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:28.081 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:28.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-16T06:50:28.426 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 8 to deep-scrub 2024-09-16T06:50:28.445 INFO:tasks.ceph:Scrubbing osd.12 2024-09-16T06:50:28.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:28.602 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:28.602 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:28.602 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:28.618 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-09-16T06:50:28.901 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 12 to deep-scrub 2024-09-16T06:50:28.918 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T06:50:28.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:29.078 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:29.078 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:29.079 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:29.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T06:50:29.372 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T06:50:29.390 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T06:50:29.391 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:29.546 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:29.546 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:29.546 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:29.561 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T06:50:29.831 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T06:50:29.848 INFO:tasks.ceph:Scrubbing osd.9 2024-09-16T06:50:29.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:29.993 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:29.993 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:29.993 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:30.008 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-16T06:50:30.290 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 9 to deep-scrub 2024-09-16T06:50:30.307 INFO:tasks.ceph:Scrubbing osd.13 2024-09-16T06:50:30.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:30.456 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:30.456 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:30.456 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:30.470 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-09-16T06:50:30.754 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 13 to deep-scrub 2024-09-16T06:50:30.773 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T06:50:30.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:30.933 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:30.933 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:30.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T06:50:31.230 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T06:50:31.248 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T06:50:31.248 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:31.397 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:31.398 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:31.398 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:31.414 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T06:50:31.697 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T06:50:31.715 INFO:tasks.ceph:Scrubbing osd.10 2024-09-16T06:50:31.715 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:31.864 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:31.864 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:31.864 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:31.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-16T06:50:32.148 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 10 to deep-scrub 2024-09-16T06:50:32.166 INFO:tasks.ceph:Scrubbing osd.14 2024-09-16T06:50:32.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:32.324 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:32.325 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:32.325 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:32.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-09-16T06:50:32.640 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 14 to deep-scrub 2024-09-16T06:50:32.659 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T06:50:32.659 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:32.816 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:32.816 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:32.816 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:32.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T06:50:33.123 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T06:50:33.142 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T06:50:33.142 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:33.290 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:33.290 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:33.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T06:50:33.586 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T06:50:33.603 INFO:tasks.ceph:Scrubbing osd.11 2024-09-16T06:50:33.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:33.761 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:33.761 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:33.761 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:33.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-16T06:50:34.064 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 11 to deep-scrub 2024-09-16T06:50:34.084 INFO:tasks.ceph:Scrubbing osd.15 2024-09-16T06:50:34.084 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-09-16T06:50:34.237 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-16T06:50:34.237 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T06:50:34.237 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-16T06:50:34.252 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-09-16T06:50:34.529 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 15 to deep-scrub 2024-09-16T06:50:34.548 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:34.860 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:34.861 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:34.878 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-16T06:50:33.793997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":590532,"kb_used_data":36920,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509358908,"statfs":{"total":1546188226560,"available":1545583521792,"internally_reserved":0,"allocated":37806080,"data_stored":22617398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002285"},"pg_stats":[{"pgid":"1.0","version":"28'157","reported_seq":340,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T06:50:27.752320+0000","last_change":"2024-09-16T06:49:26.823490+0000","last_active":"2024-09-16T06:50:27.752320+0000","last_peered":"2024-09-16T06:50:27.752320+0000","last_clean":"2024-09-16T06:50:27.752320+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:27.752320+0000","last_undegraded":"2024-09-16T06:50:27.752320+0000","last_fullsized":"2024-09-16T06:50:27.752320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'157","last_scrub_stamp":"2024-09-16T06:49:26.823237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:06.616578+0000","last_clean_scrub_stamp":"2024-09-16T06:49:26.823237+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:36.510558+0000","scrub_duration":0.0041744199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"26'1","reported_seq":145,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T06:50:27.489529+0000","last_change":"2024-09-16T06:49:36.234569+0000","last_active":"2024-09-16T06:50:27.489529+0000","last_peered":"2024-09-16T06:50:27.489529+0000","last_clean":"2024-09-16T06:50:27.489529+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:27.489529+0000","last_undegraded":"2024-09-16T06:50:27.489529+0000","last_fullsized":"2024-09-16T06:50:27.489529+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'1","last_scrub_stamp":"2024-09-16T06:49:36.234276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:36.234276+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:44.146105+0000","scrub_duration":0.0038674949999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":145,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.757882+0000","last_change":"2024-09-16T06:49:24.349814+0000","last_active":"2024-09-16T06:50:21.757882+0000","last_peered":"2024-09-16T06:50:21.757882+0000","last_clean":"2024-09-16T06:50:21.757882+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:21.757882+0000","last_undegraded":"2024-09-16T06:50:21.757882+0000","last_fullsized":"2024-09-16T06:50:21.757882+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:24.349587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:24.349587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:41.790667+0000","scrub_duration":0.0035253659999999998,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T06:50:27.489525+0000","last_change":"2024-09-16T06:49:39.164006+0000","last_active":"2024-09-16T06:50:27.489525+0000","last_peered":"2024-09-16T06:50:27.489525+0000","last_clean":"2024-09-16T06:50:27.489525+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:27.489525+0000","last_undegraded":"2024-09-16T06:50:27.489525+0000","last_fullsized":"2024-09-16T06:50:27.489525+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:39.163830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.163830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:50:50.850409+0000","scrub_duration":0.0052168600000000002,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"28'2","reported_seq":147,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-16T06:50:21.757895+0000","last_change":"2024-09-16T06:49:39.238192+0000","last_active":"2024-09-16T06:50:21.757895+0000","last_peered":"2024-09-16T06:50:21.757895+0000","last_clean":"2024-09-16T06:50:21.757895+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:21.757895+0000","last_undegraded":"2024-09-16T06:50:21.757895+0000","last_fullsized":"2024-09-16T06:50:21.757895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2024-09-16T06:49:39.237948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:39.237948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:04.804819+0000","scrub_duration":0.0076161809999999996,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":157,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T06:50:30.335895+0000","last_change":"2024-09-16T06:50:30.335895+0000","last_active":"2024-09-16T06:50:30.335895+0000","last_peered":"2024-09-16T06:50:30.335895+0000","last_clean":"2024-09-16T06:50:30.335895+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:30.335895+0000","last_undegraded":"2024-09-16T06:50:30.335895+0000","last_fullsized":"2024-09-16T06:50:30.335895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:30.335655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:30.335655+0000","last_clean_scrub_stamp":"2024-09-16T06:50:30.335655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:35.700038+0000","scrub_duration":0.0030859899999999998,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T06:50:28.432819+0000","last_change":"2024-09-16T06:49:50.812998+0000","last_active":"2024-09-16T06:50:28.432819+0000","last_peered":"2024-09-16T06:50:28.432819+0000","last_clean":"2024-09-16T06:50:28.432819+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:28.432819+0000","last_undegraded":"2024-09-16T06:50:28.432819+0000","last_fullsized":"2024-09-16T06:50:28.432819+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:49:50.812756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:49:50.812756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0062089459999999999,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T06:50:31.969331+0000","last_change":"2024-09-16T06:50:31.969518+0000","last_active":"2024-09-16T06:50:31.969331+0000","last_peered":"2024-09-16T06:50:31.969331+0000","last_clean":"2024-09-16T06:50:31.969331+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:31.969331+0000","last_undegraded":"2024-09-16T06:50:31.969331+0000","last_fullsized":"2024-09-16T06:50:31.969331+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:31.969282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:31.969282+0000","last_clean_scrub_stamp":"2024-09-16T06:50:31.969282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:37.854543+0000","scrub_duration":0.0027412970000000002,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T06:50:27.969978+0000","last_change":"2024-09-16T06:50:20.056944+0000","last_active":"2024-09-16T06:50:27.969978+0000","last_peered":"2024-09-16T06:50:27.969978+0000","last_clean":"2024-09-16T06:50:27.969978+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:27.969978+0000","last_undegraded":"2024-09-16T06:50:27.969978+0000","last_fullsized":"2024-09-16T06:50:27.969978+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:20.056663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:48:08.638478+0000","last_clean_scrub_stamp":"2024-09-16T06:50:20.056663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:34.570002+0000","scrub_duration":0.0034361130000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247840,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33360,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":1.0429999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":1.1120000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":1.204},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.161}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.0920000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0780000000000001,"15min":1.093},"min":{"1min":0.67300000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.381,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":25.283000000000001,"5min":8.7789999999999999,"15min":6.0289999999999999},"min":{"1min":0.73699999999999999,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":412.197,"5min":412.197,"15min":412.197},"last":1.016}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":1.369}]},{"osd":13,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.28,"15min":1.2529999999999999},"min":{"1min":1.143,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.407},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2869999999999999,"15min":1.294},"min":{"1min":0.95399999999999996,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5229999999999999,"5min":1.839,"15min":1.839},"last":1.141}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.3009999999999999}]}]},{"osd":14,"up_from":23,"seq":98784247842,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51408,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320432,"statfs":{"total":96636764160,"available":96584122368,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.359}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":1.1839999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":0.88400000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0029999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.387},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.3009999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.409}]}]},{"osd":11,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51412,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320428,"statfs":{"total":96636764160,"available":96584118272,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":0.753},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.262}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":0.97799999999999998}]},{"osd":3,"last update":"Mon Sep 16 06:50:32 2024","interfaces":[{"interface":"back","average":{"1min":29.847999999999999,"5min":29.847999999999999,"15min":29.847999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":577.17399999999998,"5min":577.17399999999998,"15min":577.17399999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.044}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.198},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":1.099}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":33.334000000000003,"15min":38.677999999999997},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.0459999999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.143},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.24,"15min":1.2310000000000001},"min":{"1min":0.79600000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.782,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.4219999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.238}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.458},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.3520000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.397},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.512}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.288},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.375}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33348,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338492,"statfs":{"total":96636764160,"available":96602615808,"internally_reserved":0,"allocated":2285568,"data_stored":1338955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":0.67200000000000004},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":477.90199999999999}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.147},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.123}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.1990000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.74099999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":627.13199999999995},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.05}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":15,"last update":"Mon Sep 16 06:50:21 2024","interfaces":[{"interface":"back","average":{"1min":13.247,"5min":4.0030000000000001,"15min":2.4620000000000002},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":272.404,"5min":272.404,"15min":272.404},"last":1.341},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.615,"15min":1.6419999999999999},"min":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.9890000000000001,"5min":2.343,"15min":2.343},"last":1.177}]}]},{"osd":13,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33344,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2281472,"data_stored":1338936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.72999999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.0980000000000001},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":1.002}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.212}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":0.76800000000000002}]},{"osd":7,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.3600000000000001,"15min":1.4159999999999999},"min":{"1min":0.66500000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.444,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.413,"15min":1.4650000000000001},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.88600000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.1319999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.5,"15min":1.5349999999999999},"min":{"1min":0.68500000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8340000000000001,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.385},{"interface":"front","average":{"1min":1.3140000000000001,"5min":14.577999999999999,"15min":16.789000000000001},"min":{"1min":0.85899999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.752,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.3580000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.1850000000000001},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.163}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.248}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.3089999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247842,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2830336,"data_stored":1877616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.82699999999999996}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":0.66200000000000003},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.628},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.97899999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.87},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 06:50:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.2250000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.04},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":0.76900000000000002}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.175},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":0.80600000000000005}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.69999999999999996}]}]},{"osd":4,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:27 2024","interfaces":[{"interface":"back","average":{"1min":14.571,"5min":14.571,"15min":14.571},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":194.00999999999999,"5min":194.00999999999999,"15min":194.00999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.5}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":446.35199999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":20.175000000000001,"5min":20.175000000000001,"15min":20.175000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.437},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.40799999999999997}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":0.63900000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":0.628},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":0.755}]},{"osd":11,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.80500000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":34.957000000000001,"5min":34.957000000000001,"15min":34.957000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":502.23700000000002,"5min":502.23700000000002,"15min":502.23700000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.74399999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":0.68700000000000006}]}]},{"osd":5,"up_from":115,"seq":493921239051,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30864,"kb_used_data":2244,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94340976,"statfs":{"total":96636764160,"available":96605159424,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]}]},{"osd":6,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2830336,"data_stored":1877616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.81299999999999994}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.2030000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":0.997},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.0369999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.405},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.274}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.238},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.071}]},{"osd":8,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.329},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.3520000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.476}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.589},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.706}]},{"osd":14,"last update":"Mon Sep 16 06:50:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.6739999999999999},{"interface":"front","average":{"1min":16.689,"5min":16.689,"15min":16.689},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":259.19600000000003,"5min":259.19600000000003,"15min":259.19600000000003},"last":1.629}]},{"osd":15,"last update":"Mon Sep 16 06:50:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.393,"15min":1.365},"min":{"1min":1.077,"5min":0.753,"15min":0.753},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.341,"15min":1.323},"min":{"1min":1.1060000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7849999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.5109999999999999}]}]},{"osd":7,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.76900000000000002,"15min":0.76000000000000001},"min":{"1min":0.59699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":10.544,"15min":12.159000000000001},"min":{"1min":0.58499999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1639999999999999,"5min":305.589,"15min":305.589},"last":0.96699999999999997}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":0.746},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.82799999999999996}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.877}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.91600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":1.367},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.2589999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":1.032}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.2909999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.177},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.008}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.415},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.444}]}]},{"osd":8,"up_from":22,"seq":94489280545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33364,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338476,"statfs":{"total":96636764160,"available":96602599424,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":5.5460000000000003,"5min":5.5460000000000003,"15min":5.5460000000000003},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.77200000000000002}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.73199999999999998}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.86599999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":0.66500000000000004}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.0049999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.389,"15min":1.423},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":3.169,"15min":3.169},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.359,"15min":1.405},"min":{"1min":0.76200000000000001,"5min":0.75,"15min":0.75},"max":{"1min":1.7729999999999999,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":0.98899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.024}]},{"osd":14,"last update":"Mon Sep 16 06:50:23 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.3029999999999999,"15min":1.327},"min":{"1min":0.78300000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0539999999999998,"5min":2.399,"15min":2.399},"last":1.05},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.6659999999999999,"15min":1.726},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1539999999999999,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.1930000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":0.91100000000000003}]}]},{"osd":12,"up_from":22,"seq":94489280546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33364,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338476,"statfs":{"total":96636764160,"available":96602599424,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.37}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.54200000000000004},{"interface":"front","average":{"1min":37.573,"5min":37.573,"15min":37.573},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":692.62900000000002,"5min":692.62900000000002,"15min":692.62900000000002},"last":0.46200000000000002}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:50:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.022},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5309999999999999,"5min":1.554,"15min":1.554},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.591,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.70399999999999996}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":0.68000000000000005}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":0.79300000000000004},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":0.73499999999999999}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.72999999999999998}]},{"osd":15,"last update":"Mon Sep 16 06:50:21 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":26.803000000000001,"15min":31.056999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8759999999999999,"5min":575.04499999999996,"15min":575.04499999999996},"last":0.751},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.329,"15min":1.3400000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6950000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.67100000000000004}]}]},{"osd":1,"up_from":23,"seq":98784247843,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":0.998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":0.95799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.91000000000000003}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":1.357},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.034}]},{"osd":10,"last update":"Mon Sep 16 06:50:25 2024","interfaces":[{"interface":"back","average":{"1min":6.3209999999999997,"5min":2.2330000000000001,"15min":1.5509999999999999},"min":{"1min":0.88500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":108.11799999999999,"5min":108.11799999999999,"15min":108.11799999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.1499999999999999,"15min":1.141},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.101}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.208}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.1719999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:25 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.92600000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.633,"5min":1.78,"15min":1.78},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.417,"15min":1.4099999999999999},"min":{"1min":1.24,"5min":1,"15min":1},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.246}]},{"osd":14,"last update":"Mon Sep 16 06:50:28 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4019999999999999,"15min":1.3859999999999999},"min":{"1min":1.222,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":29.141999999999999,"5min":6.9619999999999997,"15min":3.266},"min":{"1min":1.2769999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":525.96799999999996,"5min":525.96799999999996,"15min":525.96799999999996},"last":1.2749999999999999}]}]},{"osd":2,"up_from":23,"seq":98784247841,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":0.99399999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.335},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":1.024}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.81699999999999995}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.054},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":0.88800000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":0.93600000000000005}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.3999999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.454},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.081}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":0.97299999999999998}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.129},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.5649999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.6040000000000001}]}]},{"osd":9,"up_from":23,"seq":98784247840,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33364,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338476,"statfs":{"total":96636764160,"available":96602599424,"internally_reserved":0,"allocated":2301952,"data_stored":1348707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":0.93799999999999994}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.044}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.89100000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.077},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2589999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.125},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":0.98699999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.014},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.304}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.3440000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":571.77599999999995}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.391},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.236}]}]},{"osd":10,"up_from":23,"seq":98784247841,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51408,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320432,"statfs":{"total":96636764160,"available":96584122368,"internally_reserved":0,"allocated":2297856,"data_stored":1348688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":29.286999999999999,"5min":6.4790000000000001,"15min":2.677},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":599.428,"5min":599.428,"15min":599.428},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.86599999999999999,"15min":0.85199999999999998},"min":{"1min":0.51900000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.90300000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.97699999999999998,"15min":0.97799999999999998},"min":{"1min":0.71199999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.101,"5min":87.593000000000004,"15min":102.008},"min":{"1min":0.748,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.401,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.4219999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0429999999999999,"15min":1.0349999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.395,"5min":1.516,"15min":1.516},"last":0.72099999999999997}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":705.91800000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.2490000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1080000000000001,"15min":1.0960000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1899999999999999,"15min":1.194},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.28}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.1799999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.022},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.474}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":1.387},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":1.1459999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.5},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.218}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.526},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.5449999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:34.885 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T06:49:26.823237+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=49, tm_sec=26, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.885 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T06:49:36.234276+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=49, tm_sec=36, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.886 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T06:49:24.349587+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=49, tm_sec=24, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.886 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T06:49:39.163830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=49, tm_sec=39, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.886 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T06:49:39.237948+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=49, tm_sec=39, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.886 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T06:49:50.812756+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=49, tm_sec=50, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.886 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T06:50:20.056663+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=6, tm_min=50, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T06:50:34.886 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T06:50:54.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T06:50:55.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T06:50:55.238+0000 7f34477fe640 0 --2- 172.21.15.100:0/2082639000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344814d240 0x7f344806ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T06:50:55.462 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T06:50:55.463 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-16T06:50:55.482 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-16T06:50:53.799934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":205,"num_write_kb":3462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":591624,"kb_used_data":37992,"kb_used_omap":36,"kb_used_meta":553371,"kb_avail":1509357816,"statfs":{"total":1546188226560,"available":1545582403584,"internally_reserved":0,"allocated":38903808,"data_stored":23264282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37722,"internal_metadata":566652070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001796"},"pg_stats":[{"pgid":"1.0","version":"150'159","reported_seq":358,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:48.734121+0000","last_change":"2024-09-16T06:50:35.986441+0000","last_active":"2024-09-16T06:50:48.734121+0000","last_peered":"2024-09-16T06:50:48.734121+0000","last_clean":"2024-09-16T06:50:48.734121+0000","last_became_active":"2024-09-16T06:48:08.412616+0000","last_became_peered":"2024-09-16T06:48:08.412616+0000","last_unstale":"2024-09-16T06:50:48.734121+0000","last_undegraded":"2024-09-16T06:50:48.734121+0000","last_fullsized":"2024-09-16T06:50:48.734121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'159","last_scrub_stamp":"2024-09-16T06:50:35.986162+0000","last_deep_scrub":"150'159","last_deep_scrub_stamp":"2024-09-16T06:50:35.986162+0000","last_clean_scrub_stamp":"2024-09-16T06:50:35.986162+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:03.285504+0000","scrub_duration":0.010778106000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"150'2","reported_seq":162,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:48.734258+0000","last_change":"2024-09-16T06:50:34.211271+0000","last_active":"2024-09-16T06:50:48.734258+0000","last_peered":"2024-09-16T06:50:48.734258+0000","last_clean":"2024-09-16T06:50:48.734258+0000","last_became_active":"2024-09-16T06:48:09.664154+0000","last_became_peered":"2024-09-16T06:48:09.664154+0000","last_unstale":"2024-09-16T06:50:48.734258+0000","last_undegraded":"2024-09-16T06:50:48.734258+0000","last_fullsized":"2024-09-16T06:50:48.734258+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'2","last_scrub_stamp":"2024-09-16T06:50:34.211033+0000","last_deep_scrub":"150'2","last_deep_scrub_stamp":"2024-09-16T06:50:34.211033+0000","last_clean_scrub_stamp":"2024-09-16T06:50:34.211033+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:51.749319+0000","scrub_duration":0.0064817379999999999,"objects_trimmed":0,"snaptrim_duration":0.00061519400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,15],"acting":[14,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T06:50:47.655447+0000","last_change":"2024-09-16T06:50:44.297811+0000","last_active":"2024-09-16T06:50:47.655447+0000","last_peered":"2024-09-16T06:50:47.655447+0000","last_clean":"2024-09-16T06:50:47.655447+0000","last_became_active":"2024-09-16T06:48:09.853852+0000","last_became_peered":"2024-09-16T06:48:09.853852+0000","last_unstale":"2024-09-16T06:50:47.655447+0000","last_undegraded":"2024-09-16T06:50:47.655447+0000","last_fullsized":"2024-09-16T06:50:47.655447+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:44.297522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:44.297522+0000","last_clean_scrub_stamp":"2024-09-16T06:50:44.297522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:53.813427+0000","scrub_duration":0.0037032010000000001,"objects_trimmed":0,"snaptrim_duration":0.00062835600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:48.734234+0000","last_change":"2024-09-16T06:50:39.173387+0000","last_active":"2024-09-16T06:50:48.734234+0000","last_peered":"2024-09-16T06:50:48.734234+0000","last_clean":"2024-09-16T06:50:48.734234+0000","last_became_active":"2024-09-16T06:48:09.665263+0000","last_became_peered":"2024-09-16T06:48:09.665263+0000","last_unstale":"2024-09-16T06:50:48.734234+0000","last_undegraded":"2024-09-16T06:50:48.734234+0000","last_fullsized":"2024-09-16T06:50:48.734234+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:39.173142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:39.173142+0000","last_clean_scrub_stamp":"2024-09-16T06:50:39.173142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:45.440344+0000","scrub_duration":0.003393194,"objects_trimmed":0,"snaptrim_duration":0.00066927000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1],"acting":[14,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"151'3","reported_seq":164,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T06:50:47.655399+0000","last_change":"2024-09-16T06:50:40.320273+0000","last_active":"2024-09-16T06:50:47.655399+0000","last_peered":"2024-09-16T06:50:47.655399+0000","last_clean":"2024-09-16T06:50:47.655399+0000","last_became_active":"2024-09-16T06:48:09.666578+0000","last_became_peered":"2024-09-16T06:48:09.666578+0000","last_unstale":"2024-09-16T06:50:47.655399+0000","last_undegraded":"2024-09-16T06:50:47.655399+0000","last_fullsized":"2024-09-16T06:50:47.655399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'3","last_scrub_stamp":"2024-09-16T06:50:40.320021+0000","last_deep_scrub":"151'3","last_deep_scrub_stamp":"2024-09-16T06:50:40.320021+0000","last_clean_scrub_stamp":"2024-09-16T06:50:40.320021+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:06.048752+0000","scrub_duration":0.0065344469999999997,"objects_trimmed":0,"snaptrim_duration":0.0022663560000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9],"acting":[3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":161,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:48.823236+0000","last_change":"2024-09-16T06:50:30.335895+0000","last_active":"2024-09-16T06:50:48.823236+0000","last_peered":"2024-09-16T06:50:48.823236+0000","last_clean":"2024-09-16T06:50:48.823236+0000","last_became_active":"2024-09-16T06:48:09.669530+0000","last_became_peered":"2024-09-16T06:48:09.669530+0000","last_unstale":"2024-09-16T06:50:48.823236+0000","last_undegraded":"2024-09-16T06:50:48.823236+0000","last_fullsized":"2024-09-16T06:50:48.823236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:30.335655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:30.335655+0000","last_clean_scrub_stamp":"2024-09-16T06:50:30.335655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:35.700038+0000","scrub_duration":0.0030859899999999998,"objects_trimmed":0,"snaptrim_duration":0.00060903200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,9],"acting":[1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":178,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:48.737527+0000","last_change":"2024-09-16T06:50:31.766195+0000","last_active":"2024-09-16T06:50:48.737527+0000","last_peered":"2024-09-16T06:50:48.737527+0000","last_clean":"2024-09-16T06:50:48.737527+0000","last_became_active":"2024-09-16T06:49:49.972089+0000","last_became_peered":"2024-09-16T06:49:49.972089+0000","last_unstale":"2024-09-16T06:50:48.737527+0000","last_undegraded":"2024-09-16T06:50:48.737527+0000","last_fullsized":"2024-09-16T06:50:48.737527+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:31.766072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:31.766072+0000","last_clean_scrub_stamp":"2024-09-16T06:50:31.766072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:52.412100+0000","scrub_duration":0.002413531,"objects_trimmed":0,"snaptrim_duration":0.00044935599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:48.016531+0000","last_change":"2024-09-16T06:50:31.969518+0000","last_active":"2024-09-16T06:50:48.016531+0000","last_peered":"2024-09-16T06:50:48.016531+0000","last_clean":"2024-09-16T06:50:48.016531+0000","last_became_active":"2024-09-16T06:48:09.856354+0000","last_became_peered":"2024-09-16T06:48:09.856354+0000","last_unstale":"2024-09-16T06:50:48.016531+0000","last_undegraded":"2024-09-16T06:50:48.016531+0000","last_fullsized":"2024-09-16T06:50:48.016531+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:31.969282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:31.969282+0000","last_clean_scrub_stamp":"2024-09-16T06:50:31.969282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:37.854543+0000","scrub_duration":0.0027412970000000002,"objects_trimmed":0,"snaptrim_duration":0.00063479900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":173,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T06:50:47.123309+0000","last_change":"2024-09-16T06:50:35.962531+0000","last_active":"2024-09-16T06:50:47.123309+0000","last_peered":"2024-09-16T06:50:47.123309+0000","last_clean":"2024-09-16T06:50:47.123309+0000","last_became_active":"2024-09-16T06:48:09.895088+0000","last_became_peered":"2024-09-16T06:48:09.895088+0000","last_unstale":"2024-09-16T06:50:47.123309+0000","last_undegraded":"2024-09-16T06:50:47.123309+0000","last_fullsized":"2024-09-16T06:50:47.123309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T06:50:35.962282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T06:50:35.962282+0000","last_clean_scrub_stamp":"2024-09-16T06:50:35.962282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:04.051364+0000","scrub_duration":0.0036470690000000002,"objects_trimmed":0,"snaptrim_duration":0.00062087899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,10],"acting":[15,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":23,"seq":98784247844,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33424,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,7,8,9,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":36.363,"15min":42.286999999999999},"min":{"1min":0.503,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.351,"5min":780.85000000000002,"15min":780.85000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":1,"5min":7.9660000000000002,"15min":9.1270000000000007},"min":{"1min":0.58199999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6819999999999999,"5min":164.398,"15min":164.398},"last":0.84499999999999997}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.0409999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4139999999999999,"5min":1.522,"15min":1.522},"last":152.06},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0800000000000001,"15min":1.0740000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.51,"5min":1.633,"15min":1.633},"last":0.76200000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0549999999999999,"15min":1.0680000000000001},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4399999999999999,"5min":1.675,"15min":1.675},"last":0.80400000000000005},{"interface":"front","average":{"1min":50.198999999999998,"5min":10.916,"15min":4.3689999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":838.50999999999999,"5min":838.50999999999999,"15min":838.50999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":4,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0620000000000001},"min":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.752},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.72399999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3320000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":30.042999999999999,"5min":16.416,"15min":14.145},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":465.565,"5min":465.565,"15min":465.565},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.194,"15min":1.2070000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.651,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.1619999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0780000000000001,"15min":1.093},"min":{"1min":0.67300000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.381,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":25.283000000000001,"5min":8.7789999999999999,"15min":6.0289999999999999},"min":{"1min":0.73699999999999999,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":412.197,"5min":412.197,"15min":412.197},"last":1.218}]},{"osd":9,"last update":"Mon Sep 16 06:50:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.194},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.075}]},{"osd":10,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":21.707999999999998,"5min":5.5270000000000001,"15min":2.8300000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":348.30599999999998,"5min":348.30599999999998,"15min":348.30599999999998},"last":1.113},{"interface":"front","average":{"1min":1.3120000000000001,"5min":9.9199999999999999,"15min":11.355},"min":{"1min":0.76700000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.7809999999999999,"5min":197.559,"15min":197.559},"last":0.98599999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.28,"15min":1.2529999999999999},"min":{"1min":1.143,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.135},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2869999999999999,"15min":1.294},"min":{"1min":0.95399999999999996,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.5229999999999999,"5min":1.839,"15min":1.839},"last":1.024}]},{"osd":14,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3660000000000001,"15min":1.387},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.605,"5min":1.95,"15min":1.95},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.425,"15min":1.429},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1739999999999999,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.2729999999999999}]}]},{"osd":14,"up_from":23,"seq":98784247846,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.244},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":21.920000000000002,"15min":25.420000000000002},"min":{"1min":0.69899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":525.66800000000001,"15min":525.66800000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":5.8579999999999997,"5min":1.8819999999999999,"15min":1.2190000000000001},"min":{"1min":0.55300000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":105.913,"5min":105.913,"15min":105.913},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0429999999999999,"15min":1.0409999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.472,"5min":1.873,"15min":1.873},"last":0.996},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.048,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.6479999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.92200000000000004}]},{"osd":4,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.012,"15min":1.002},"min":{"1min":0.65700000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5860000000000001,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92900000000000005,"15min":0.92300000000000004},"min":{"1min":0.65700000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0660000000000001,"15min":1.071},"min":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.121,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.736,"5min":2.883,"15min":2.883},"last":1.216}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0940000000000001,"15min":1.099},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.294},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.0289999999999999},"min":{"1min":0.72399999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4630000000000001,"5min":1.748,"15min":1.748},"last":1.0820000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:50:33 2024","interfaces":[{"interface":"back","average":{"1min":3.9990000000000001,"5min":3.9990000000000001,"15min":3.9990000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":58.156999999999996,"5min":58.156999999999996,"15min":58.156999999999996},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.1579999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.353},{"interface":"front","average":{"1min":13.052,"5min":13.052,"15min":13.052},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":277.46800000000002,"5min":277.46800000000002,"15min":277.46800000000002},"last":1.2290000000000001}]},{"osd":11,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.395},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.4119999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:22 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1359999999999999,"15min":1.1160000000000001},"min":{"1min":0.86099999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.625,"5min":1.782,"15min":1.782},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2589999999999999,"15min":1.2490000000000001},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7130000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.3600000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":17.013000000000002,"5min":4.5330000000000004,"15min":2.4529999999999998},"min":{"1min":0.82599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":330.37700000000001,"5min":330.37700000000001,"15min":330.37700000000001},"last":1.387},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.282,"15min":1.272},"min":{"1min":0.81200000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.617,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.3380000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":15.686,"15min":18.077999999999999},"min":{"1min":0.85699999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6559999999999999,"5min":360.42200000000003,"15min":360.42200000000003},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.48,"15min":1.49},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.758,"5min":3.7709999999999999,"15min":3.7709999999999999},"last":1.3169999999999999}]}]},{"osd":11,"up_from":23,"seq":98784247845,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51488,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2379776,"data_stored":1396948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":49.518000000000001,"5min":50.578000000000003,"15min":50.755000000000003},"min":{"1min":0.64500000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":924.61599999999999,"5min":949.71199999999999,"15min":949.71199999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1819999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":17.111999999999998,"5min":4.2460000000000004,"15min":2.1019999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":321.70100000000002,"5min":321.70100000000002,"15min":321.70100000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":38.716000000000001,"15min":44.999000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.7150000000000001,"5min":894.49900000000002,"15min":894.49900000000002},"last":1.244}]},{"osd":2,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":42.258000000000003,"5min":9.327,"15min":3.839},"min":{"1min":0.749,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":781.33199999999999,"5min":781.33199999999999,"15min":781.33199999999999},"last":1.27},{"interface":"front","average":{"1min":1.113,"5min":4.1210000000000004,"15min":4.6230000000000002},"min":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.821,"5min":71.503,"15min":71.503},"last":0.79100000000000004}]},{"osd":3,"last update":"Mon Sep 16 06:50:32 2024","interfaces":[{"interface":"back","average":{"1min":29.847999999999999,"5min":29.847999999999999,"15min":29.847999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":577.17399999999998,"5min":577.17399999999998,"15min":577.17399999999998},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.85199999999999998}]},{"osd":4,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6220000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":9.5429999999999993,"5min":2.8690000000000002,"15min":1.7569999999999999},"min":{"1min":0.63400000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":172.85499999999999,"5min":172.85499999999999,"15min":172.85499999999999},"last":0.92000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8879999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":33.334000000000003,"15min":38.677999999999997},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.0459999999999998,"5min":722.72199999999998,"15min":722.72199999999998},"last":1.109},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.24,"15min":1.2310000000000001},"min":{"1min":0.79600000000000004,"5min":0.751,"15min":0.751},"max":{"1min":1.782,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.3999999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.254,"15min":1.27},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.78,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.4239999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3240000000000001,"15min":1.3360000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.861,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.446},{"interface":"front","average":{"1min":1.228,"5min":1.26,"15min":1.266},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9379999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.2170000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.302},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.329}]},{"osd":14,"last update":"Mon Sep 16 06:50:20 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4650000000000001,"15min":1.482},"min":{"1min":0.86799999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0880000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.492,"15min":1.504},"min":{"1min":0.94099999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9279999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.3560000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":21,"seq":90194313253,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2351104,"data_stored":1377663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.456,"5min":1.502,"15min":1.502},"last":0.97099999999999997},{"interface":"front","average":{"1min":36.066000000000003,"5min":8.0990000000000002,"15min":3.4380000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":841.71299999999997,"5min":841.71299999999997,"15min":841.71299999999997},"last":1.2050000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":2,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3819999999999999,"5min":1.611,"15min":1.611},"last":0.627},{"interface":"front","average":{"1min":1.105,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.69499999999999995,"5min":0.626,"15min":0.626},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":47.539999999999999,"5min":10.430999999999999,"15min":4.2460000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":830.23299999999995,"5min":830.23299999999995,"15min":830.23299999999995},"last":1.3},{"interface":"front","average":{"1min":1.034,"5min":1.165,"15min":1.1870000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.032}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Mon Sep 16 06:49:11 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.0549999999999999},{"interface":"front","average":{"1min":35.844999999999999,"5min":35.844999999999999,"15min":35.844999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":622.61099999999999,"5min":622.61099999999999,"15min":622.61099999999999},"last":1.7669999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.74},{"interface":"front","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.871}]},{"osd":8,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":30.584,"5min":30.584,"15min":30.584},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":588.24099999999999,"5min":588.24099999999999,"15min":588.24099999999999},"last":1.165},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.204}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3600000000000001,"15min":1.3959999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8009999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.363,"15min":1.3959999999999999},"min":{"1min":0.76700000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.512,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.2350000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":34.542000000000002,"15min":40.088999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.1099999999999999,"5min":502.57400000000001,"15min":502.57400000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.278,"5min":33.064999999999998,"15min":38.363},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.927,"5min":502.62200000000001,"15min":502.62200000000001},"last":1.177}]},{"osd":11,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.599},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.5569999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.516,"15min":1.552},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.786,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.569,"15min":1.605},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.347}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.286},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.375}]},{"osd":14,"last update":"Mon Sep 16 06:50:32 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.155},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.329}]},{"osd":15,"last update":"Mon Sep 16 06:50:21 2024","interfaces":[{"interface":"back","average":{"1min":13.247,"5min":4.0030000000000001,"15min":2.4620000000000002},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":272.404,"5min":272.404,"15min":272.404},"last":1.498},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.615,"15min":1.6419999999999999},"min":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.9890000000000001,"5min":2.343,"15min":2.343},"last":1.399}]}]},{"osd":13,"up_from":21,"seq":90194313253,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33424,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,6,7,8,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.94999999999999996,"15min":0.96499999999999997},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":4.2089999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.107,"5min":1.1599999999999999,"15min":1.169},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.3129999999999997,"5min":8.6449999999999996,"15min":8.6449999999999996},"last":0.72299999999999998}]},{"osd":2,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0720000000000001,"15min":1.0409999999999999},"min":{"1min":0.59699999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":52.232999999999997,"5min":11.278,"15min":4.452},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":970.10299999999995,"5min":970.10299999999995,"15min":970.10299999999995},"last":1.321}]},{"osd":3,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":1.534},{"interface":"front","average":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":98.950000000000003,"5min":98.950000000000003,"15min":98.950000000000003},"last":1.216}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":23.241,"15min":26.878},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":4.6600000000000001,"5min":385.83800000000002,"15min":385.83800000000002},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.1930000000000001,"15min":1.163},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.5919999999999996,"5min":4.5919999999999996,"15min":4.5919999999999996},"last":1.1599999999999999}]},{"osd":7,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.3600000000000001,"15min":1.4159999999999999},"min":{"1min":0.66500000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.444,"5min":4.5119999999999996,"15min":4.5119999999999996},"last":1.498},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.413,"15min":1.4650000000000001},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":1.411}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2310000000000001,"15min":1.2050000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":1.645},{"interface":"front","average":{"1min":1.369,"5min":4.6319999999999997,"15min":5.1760000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.5750000000000002,"5min":96.432000000000002,"15min":96.432000000000002},"last":1.3759999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.5,"15min":1.5349999999999999},"min":{"1min":0.68500000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8340000000000001,"5min":4.7160000000000002,"15min":4.7160000000000002},"last":1.571},{"interface":"front","average":{"1min":1.3140000000000001,"5min":14.577999999999999,"15min":16.789000000000001},"min":{"1min":0.85899999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.752,"5min":210.85599999999999,"15min":210.85599999999999},"last":1.6879999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.6859999999999999,"15min":1.716},"min":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.6349999999999998,"5min":10.728,"15min":10.728},"last":1.304},{"interface":"front","average":{"1min":49.472000000000001,"5min":11.013999999999999,"15min":4.6040000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":912.29999999999995,"5min":912.29999999999995,"15min":912.29999999999995},"last":1.181}]},{"osd":12,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.2949999999999999,"15min":1.2509999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.8949999999999996,"5min":4.8949999999999996,"15min":4.8949999999999996},"last":1.9379999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3759999999999999,"15min":1.341},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.7450000000000001}]},{"osd":14,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.4610000000000001,"15min":1.4279999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":4.9870000000000001,"5min":4.9870000000000001,"15min":4.9870000000000001},"last":1.716},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.4510000000000001,"15min":1.411},"min":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":4.9260000000000002},"last":1.792}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]}]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33944,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337896,"statfs":{"total":96636764160,"available":96602005504,"internally_reserved":0,"allocated":2895872,"data_stored":1916224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":31.759,"15min":36.909999999999997},"min":{"1min":0.46600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.127,"5min":734.39200000000005,"15min":734.39200000000005},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0820000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0329999999999999,"15min":1.0349999999999999},"min":{"1min":0.64300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5029999999999999,"5min":1.998,"15min":1.998},"last":1.4350000000000001},{"interface":"front","average":{"1min":18.358000000000001,"5min":4.4809999999999999,"15min":2.1680000000000001},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":297.21100000000001,"5min":297.21100000000001,"15min":297.21100000000001},"last":1.1379999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.25,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.998,"15min":1},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.008}]},{"osd":4,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.736,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Mon Sep 16 06:49:06 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.073,"15min":1.089},"min":{"1min":0.63600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4910000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.347},{"interface":"front","average":{"1min":35.755000000000003,"5min":8.0350000000000001,"15min":3.415},"min":{"1min":0.57099999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":591.98199999999997,"5min":591.98199999999997,"15min":591.98199999999997},"last":1.3120000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1910000000000001,"15min":1.1970000000000001},"min":{"1min":0.73399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.143,"5min":26.291,"15min":30.481999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7689999999999999,"5min":596.70600000000002,"15min":596.70600000000002},"last":1.385}]},{"osd":8,"last update":"Mon Sep 16 06:50:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.637},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.4119999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.234,"15min":1.242},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.9730000000000001,"5min":2.081,"15min":2.081},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.2809999999999999,"15min":1.292},"min":{"1min":0.76700000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0840000000000001,"5min":2.266,"15min":2.266},"last":1.694}]},{"osd":13,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2310000000000001,"15min":1.2470000000000001},"min":{"1min":0.63900000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3360000000000001,"15min":1.347},"min":{"1min":0.74399999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5070000000000001,"5min":2.7149999999999999,"15min":2.7149999999999999},"last":1.5800000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:50:07 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3620000000000001,"15min":1.3759999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.615},{"interface":"front","average":{"1min":1.347,"5min":1.323,"15min":1.319},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":1.7330000000000001}]}]},{"osd":4,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33424,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,3,5,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.097},{"interface":"front","average":{"1min":34.289000000000001,"5min":34.289000000000001,"15min":34.289000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":702.28300000000002,"5min":702.28300000000002,"15min":702.28300000000002},"last":0.80700000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:27 2024","interfaces":[{"interface":"back","average":{"1min":14.571,"5min":14.571,"15min":14.571},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":194.00999999999999,"5min":194.00999999999999,"15min":194.00999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.2529999999999999}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":16.407,"5min":15.101000000000001,"15min":14.882999999999999},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":342.887,"5min":342.887,"15min":342.887},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0940000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:50:48 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.012}]},{"osd":6,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":25.231999999999999,"15min":29.276},"min":{"1min":0.56299999999999994,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.45,"5min":576.29200000000003,"15min":576.29200000000003},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.999,"15min":0.98899999999999999},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":8,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":36.774999999999999,"5min":8.0510000000000002,"15min":3.2639999999999998},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":791.84000000000003,"5min":791.84000000000003,"15min":791.84000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.2130000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7809999999999999,"15min":1.9199999999999999},"min":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3939999999999999,"5min":20.283999999999999,"15min":20.283999999999999},"last":1.242},{"interface":"front","average":{"1min":27.788,"5min":18.628,"15min":17.102},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":591.27999999999997,"5min":591.27999999999997,"15min":591.27999999999997},"last":1.2969999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2150000000000001,"15min":1.2250000000000001},"min":{"1min":0.75600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5069999999999999,"5min":1.542,"15min":1.542},"last":1.3380000000000001},{"interface":"front","average":{"1min":34.395000000000003,"5min":8.5709999999999997,"15min":4.2670000000000003},"min":{"1min":0.77100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":501.60700000000003,"5min":501.60700000000003,"15min":501.60700000000003},"last":1.3939999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:33 2024","interfaces":[{"interface":"back","average":{"1min":33.213000000000001,"5min":7.5460000000000003,"15min":3.2679999999999998},"min":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":706.76199999999994,"5min":706.76199999999994,"15min":706.76199999999994},"last":1.577},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.76100000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.425}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":14,"last update":"Mon Sep 16 06:50:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":28.193000000000001,"15min":32.701999999999998},"min":{"1min":0.72499999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.488,"5min":502.23700000000002,"15min":502.23700000000002},"last":1.4450000000000001},{"interface":"front","average":{"1min":5.9870000000000001,"5min":2.1709999999999998,"15min":1.5349999999999999},"min":{"1min":0.74399999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":106.32299999999999,"5min":106.32299999999999,"15min":106.32299999999999},"last":1.6319999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:09 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.248,"15min":1.242},"min":{"1min":0.84599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.601},{"interface":"front","average":{"1min":1.2350000000000001,"5min":2.0790000000000002,"15min":2.2200000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.53,"5min":20.437999999999999,"15min":20.437999999999999},"last":1.706}]}]},{"osd":5,"up_from":115,"seq":493921239055,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30932,"kb_used_data":2308,"kb_used_omap":13,"kb_used_meta":28594,"kb_avail":94340908,"statfs":{"total":96636764160,"available":96605089792,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13812,"internal_metadata":29280780},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":48.329999999999998,"5min":48.329999999999998,"15min":48.329999999999998},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":946.07500000000005,"5min":946.07500000000005,"15min":946.07500000000005},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.099}]},{"osd":4,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.014},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1890000000000001}]},{"osd":6,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.139}]},{"osd":7,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.2390000000000001},{"interface":"front","average":{"1min":36.317,"5min":36.317,"15min":36.317},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":669,"5min":669,"15min":669},"last":0.81699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":37.460000000000001,"5min":37.460000000000001,"15min":37.460000000000001},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":723.55399999999997,"5min":723.55399999999997,"15min":723.55399999999997},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.4730000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.5569999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.532}]},{"osd":12,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"min":{"1min":1.304,"5min":1.304,"15min":1.304},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.512},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.625}]},{"osd":14,"last update":"Mon Sep 16 06:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":1.264,"5min":1.264,"15min":1.264},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4279999999999999}]}]},{"osd":6,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33952,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337888,"statfs":{"total":96636764160,"available":96601997312,"internally_reserved":0,"allocated":2895872,"data_stored":1916224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,3,4,5,7,8,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.876,"15min":0.89200000000000002},"min":{"1min":0.47999999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.0649999999999999,"5min":1.669,"15min":1.669},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.60399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8799999999999999,"5min":1.905,"15min":1.905},"last":0.85399999999999998}]},{"osd":1,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Mon Sep 16 06:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.7849999999999999,"5min":1.893,"15min":1.893},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.155,"5min":1.046,"15min":1.028},"min":{"1min":0.72199999999999998,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":1.3879999999999999}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":55.527000000000001,"5min":12.016,"15min":4.7640000000000002},"min":{"1min":0.65700000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":873.28899999999999,"5min":873.28899999999999,"15min":873.28899999999999},"last":1.528},{"interface":"front","average":{"1min":1.2470000000000001,"5min":10.872,"15min":12.476000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.6909999999999998,"5min":231.90700000000001,"15min":231.90700000000001},"last":1.286}]},{"osd":5,"last update":"Mon Sep 16 06:50:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.2490000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.278,"15min":1.262},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.143,"5min":14.808,"15min":17.085999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4510000000000001,"5min":323.55500000000001,"15min":323.55500000000001},"last":1.4970000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:50 2024","interfaces":[{"interface":"back","average":{"1min":43.561999999999998,"5min":43.561999999999998,"15min":43.561999999999998},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":892.54499999999996,"5min":892.54499999999996,"15min":892.54499999999996},"last":1.581},{"interface":"front","average":{"1min":7.5990000000000002,"5min":7.5990000000000002,"15min":7.5990000000000002},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":135.31299999999999,"5min":135.31299999999999,"15min":135.31299999999999},"last":1.5680000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.661}]},{"osd":13,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.456,"15min":1.484},"min":{"1min":0.86499999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4690000000000001,"15min":1.4650000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.714}]},{"osd":14,"last update":"Mon Sep 16 06:50:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.7490000000000001},{"interface":"front","average":{"1min":16.689,"5min":16.689,"15min":16.689},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":259.19600000000003,"5min":259.19600000000003,"15min":259.19600000000003},"last":1.772}]},{"osd":15,"last update":"Mon Sep 16 06:50:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.393,"15min":1.365},"min":{"1min":1.077,"5min":0.753,"15min":0.753},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.794},{"interface":"front","average":{"1min":1.452,"5min":1.341,"15min":1.323},"min":{"1min":1.1060000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7849999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.851}]}]},{"osd":7,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,6,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.76900000000000002,"15min":0.76000000000000001},"min":{"1min":0.59699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":10.544,"15min":12.159000000000001},"min":{"1min":0.58499999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1639999999999999,"5min":305.589,"15min":305.589},"last":0.95099999999999996}]},{"osd":1,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.59999999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.345,"5min":1.415,"15min":1.415},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":23.460000000000001,"15min":27.222000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3440000000000001,"5min":527.10299999999995,"15min":527.10299999999995},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":3.77,"5min":5.694,"15min":6.0140000000000002},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":58.692999999999998,"5min":136.72399999999999,"15min":136.72399999999999},"last":0.878},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.04}]},{"osd":3,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":16.48,"15min":19.074000000000002},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.284,"5min":508.52100000000002,"15min":508.52100000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.018,"5min":1.0069999999999999,"15min":1.0049999999999999},"min":{"1min":0.65900000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.232}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:49:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.175},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.075}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1060000000000001,"15min":1.105},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4730000000000001,"5min":1.522,"15min":1.522},"last":1.202},{"interface":"front","average":{"1min":1.171,"5min":1.1870000000000001,"15min":1.1899999999999999},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6930000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.83399999999999996}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.417,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.139,"5min":24.286000000000001,"15min":28.143999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.718,"5min":415.30099999999999,"15min":415.30099999999999},"last":1.008}]},{"osd":9,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":30.405000000000001,"15min":35.286000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5680000000000001,"5min":502.55200000000002,"15min":502.55200000000002},"last":1.147},{"interface":"front","average":{"1min":1.276,"5min":16.706,"15min":19.277999999999999},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.0070000000000001,"5min":502.39600000000002,"15min":502.39600000000002},"last":1.123}]},{"osd":10,"last update":"Mon Sep 16 06:50:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.266},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":1.105,"5min":1.105,"15min":1.105},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.3799999999999999}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":11.968999999999999,"5min":3.448,"15min":2.028},"min":{"1min":0.95099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":224.625,"5min":224.625,"15min":224.625},"last":1.304},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2789999999999999,"15min":1.2849999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.0840000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3500000000000001,"15min":1.349},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":17.440999999999999,"5min":20.393000000000001,"15min":20.885000000000002},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":339.30000000000001,"5min":514.07799999999997,"15min":514.07799999999997},"last":1.3520000000000001}]}]},{"osd":8,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,5,6,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":39.104999999999997,"5min":39.104999999999997,"15min":39.104999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":689.05700000000002,"5min":689.05700000000002,"15min":689.05700000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":8.7240000000000002,"15min":10.045},"min":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.321,"5min":177.92400000000001,"15min":177.92400000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0680000000000001,"15min":1.113},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3600000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":5,"last update":"Mon Sep 16 06:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":22.329999999999998,"15min":25.901},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3540000000000001,"5min":449.88,"15min":449.88},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88,"5min":1.367,"15min":1.448},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1950000000000001,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.872}]},{"osd":7,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.1699999999999999,"15min":1.2130000000000001},"min":{"1min":0.56899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.7410000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1200000000000001,"15min":1.1379999999999999},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.407,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1259999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.1499999999999999,"15min":1.1719999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.7190000000000001,"15min":1.841},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3979999999999999,"5min":12.366,"15min":12.366},"last":1.1060000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:17 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":45.607999999999997,"15min":53.036000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8560000000000001,"5min":998.42399999999998,"15min":998.42399999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2969999999999999,"15min":1.319},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.325}]},{"osd":11,"last update":"Mon Sep 16 06:50:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.389,"15min":1.423},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":3.169,"15min":3.169},"last":1.202},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.359,"15min":1.405},"min":{"1min":0.76200000000000001,"5min":0.75,"15min":0.75},"max":{"1min":1.7729999999999999,"5min":4.4960000000000004,"15min":4.4960000000000004},"last":1.1839999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":36.192,"5min":36.192,"15min":36.192},"last":1.151},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.355}]},{"osd":14,"last update":"Mon Sep 16 06:50:23 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.3029999999999999,"15min":1.327},"min":{"1min":0.78300000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.0539999999999998,"5min":2.399,"15min":2.399},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.6659999999999999,"15min":1.726},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1539999999999999,"5min":5.5640000000000001,"15min":5.5640000000000001},"last":1.2529999999999999}]},{"osd":15,"last update":"Mon Sep 16 06:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.669,"15min":1.7509999999999999},"min":{"1min":0.78600000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8819999999999999,"5min":5.5869999999999997,"15min":5.5869999999999997},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.423,"15min":1.456},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2,"5min":2.726,"15min":2.726},"last":1.2989999999999999}]}]},{"osd":12,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33424,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[3,4,5,6,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":43.365000000000002,"5min":43.365000000000002,"15min":43.365000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":766.10199999999998,"5min":766.10199999999998,"15min":766.10199999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Mon Sep 16 06:49:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":9.6850000000000005,"5min":2.8130000000000002,"15min":1.6679999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":151.34800000000001,"5min":151.34800000000001,"15min":151.34800000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":30.654,"15min":35.624000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3120000000000001,"5min":708.49699999999996,"15min":708.49699999999996},"last":0.94799999999999995}]},{"osd":4,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.0680000000000001,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":27.318000000000001,"15min":31.754000000000001},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.2090000000000001,"5min":630.75900000000001,"15min":630.75900000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66500000000000004},{"interface":"front","average":{"1min":40.642000000000003,"5min":40.642000000000003,"15min":40.642000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":793.73699999999997,"5min":793.73699999999997,"15min":793.73699999999997},"last":0.84199999999999997}]},{"osd":6,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.999,"15min":0.99099999999999999},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.014,"15min":1.026},"min":{"1min":0.56499999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5529999999999999,"5min":1.867,"15min":1.867},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6080000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":9,"last update":"Mon Sep 16 06:50:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.034,"15min":1.022},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5309999999999999,"5min":1.554,"15min":1.554},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.591,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.1339999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":27.678999999999998,"5min":6.5659999999999998,"15min":3.0470000000000002},"min":{"1min":0.66700000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":453.37700000000001,"5min":453.37700000000001,"15min":453.37700000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.59699999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.345,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":1.0369999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.087,"15min":1.0920000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7110000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.179},{"interface":"front","average":{"1min":9.9469999999999992,"5min":42.097000000000001,"15min":47.454999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":150.93299999999999,"5min":980.38300000000004,"15min":980.38300000000004},"last":1.012}]},{"osd":13,"last update":"Mon Sep 16 06:50:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6639999999999999,"5min":2.214,"15min":2.214},"last":0.97299999999999998},{"interface":"front","average":{"1min":15.455,"5min":4.2469999999999999,"15min":2.379},"min":{"1min":0.68999999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":229.631,"5min":229.631,"15min":229.631},"last":0.90800000000000003}]},{"osd":14,"last update":"Mon Sep 16 06:50:15 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.425,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5189999999999999,"5min":3.657,"15min":3.657},"last":1.157},{"interface":"front","average":{"1min":1.198,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":0.66100000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.829,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.2110000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:50:21 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":26.803000000000001,"15min":31.056999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.8759999999999999,"5min":575.04499999999996,"15min":575.04499999999996},"last":1.234},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.329,"15min":1.3400000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6950000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.0860000000000001}]}]},{"osd":1,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37524,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334316,"statfs":{"total":96636764160,"available":96598339584,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":9.9440000000000008,"15min":11.454000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.375,"5min":214.93299999999999,"15min":214.93299999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":2.887,"5min":1.2430000000000001,"15min":0.96899999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":42.807000000000002,"5min":42.807000000000002,"15min":42.807000000000002},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3680000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.226}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":10.132,"15min":11.641999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6519999999999999,"5min":216.958,"15min":216.958},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":28.224,"15min":32.758000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4590000000000001,"5min":645.88099999999997,"15min":645.88099999999997},"last":0.997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":8,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.149,"15min":1.157},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.419,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.1000000000000001,"15min":1.1120000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4259999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.85299999999999998}]},{"osd":9,"last update":"Mon Sep 16 06:50:14 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":23.071000000000002,"15min":26.751000000000001},"min":{"1min":0.60799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5529999999999999,"5min":493.76100000000002,"15min":493.76100000000002},"last":1.268},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5,"5min":1.675,"15min":1.675},"last":1.1319999999999999}]},{"osd":10,"last update":"Mon Sep 16 06:50:25 2024","interfaces":[{"interface":"back","average":{"1min":6.3209999999999997,"5min":2.2330000000000001,"15min":1.5509999999999999},"min":{"1min":0.88500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":108.11799999999999,"5min":108.11799999999999,"15min":108.11799999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.1499999999999999,"15min":1.141},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0349999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":30.277999999999999,"15min":35.109999999999999},"min":{"1min":0.94099999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5840000000000001,"5min":687.10799999999995,"15min":687.10799999999995},"last":1.486},{"interface":"front","average":{"1min":1.222,"5min":21.262,"15min":24.602},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.806,"5min":475.75700000000001,"15min":475.75700000000001},"last":1.1830000000000001}]},{"osd":12,"last update":"Mon Sep 16 06:50:12 2024","interfaces":[{"interface":"back","average":{"1min":18.658000000000001,"5min":4.8170000000000002,"15min":2.5099999999999998},"min":{"1min":0.93600000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":366.83499999999998,"5min":366.83499999999998,"15min":366.83499999999998},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.452,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"max":{"1min":1.538,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":1.335}]},{"osd":13,"last update":"Mon Sep 16 06:50:25 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.92600000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.633,"5min":1.78,"15min":1.78},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.417,"15min":1.4099999999999999},"min":{"1min":1.24,"5min":1,"15min":1},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.536}]},{"osd":14,"last update":"Mon Sep 16 06:50:28 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4019999999999999,"15min":1.3859999999999999},"min":{"1min":1.222,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.458},{"interface":"front","average":{"1min":29.141999999999999,"5min":6.9619999999999997,"15min":3.266},"min":{"1min":1.2769999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":525.96799999999996,"5min":525.96799999999996,"15min":525.96799999999996},"last":1.577}]}]},{"osd":2,"up_from":23,"seq":98784247845,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2363392,"data_stored":1387296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.97699999999999998,"15min":1.002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.21,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.95899999999999996,"15min":0.97899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3500000000000001,"5min":1.474,"15min":1.474},"last":1.2330000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.018,"15min":1.034},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2909999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3109999999999999,"5min":1.538,"15min":1.538},"last":1.1910000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":15.425000000000001,"15min":17.843},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1919999999999999,"5min":361.36799999999999,"15min":361.36799999999999},"last":1.304},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.131,"15min":1.1599999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2310000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.5129999999999999}]},{"osd":5,"last update":"Mon Sep 16 06:49:07 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.9239999999999999,"5min":2.9239999999999999,"15min":2.9239999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.878}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.52700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.3480000000000001,"5min":2.218,"15min":2.218},"last":1.45},{"interface":"front","average":{"1min":11.368,"5min":3.206,"15min":1.8460000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":209.36600000000001,"5min":209.36600000000001,"15min":209.36600000000001},"last":1.0900000000000001}]},{"osd":7,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":28.317,"15min":32.863999999999997},"min":{"1min":0.64100000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4199999999999999,"5min":646.98800000000006,"15min":646.98800000000006},"last":1.143},{"interface":"front","average":{"1min":1.0840000000000001,"5min":22.946000000000002,"15min":26.59},"min":{"1min":0.755,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.456,"5min":516.49000000000001,"15min":516.49000000000001},"last":1.0640000000000001}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.151,"15min":1.181},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.121},{"interface":"front","average":{"1min":1.085,"5min":1.167,"15min":1.181},"min":{"1min":0.70499999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5349999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.383}]},{"osd":9,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":12.465999999999999,"5min":3.8260000000000001,"15min":2.3860000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":228.19200000000001,"5min":228.19200000000001,"15min":228.19200000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.3779999999999999,"15min":1.4259999999999999},"min":{"1min":0.77400000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3540000000000001,"5min":3.8239999999999998,"15min":3.8239999999999998},"last":1.3460000000000001}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.47,"15min":1.5149999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5680000000000001,"5min":2.786,"15min":2.786},"last":2.1040000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.401,"5min":1.97,"15min":1.97},"last":2.1899999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4990000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.611,"15min":1.665},"min":{"1min":0.86499999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.597,"5min":3.6269999999999998,"15min":3.6269999999999998},"last":1.5980000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.5649999999999999}]}]},{"osd":9,"up_from":23,"seq":98784247844,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33428,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2367488,"data_stored":1387315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,6,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":37.814,"5min":47.905000000000001,"15min":49.587000000000003},"min":{"1min":0.753,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":736.75099999999998,"5min":941.91899999999998,"15min":941.91899999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":35.390000000000001,"5min":7.9169999999999998,"15min":3.3380000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":688.51599999999996,"5min":688.51599999999996,"15min":688.51599999999996},"last":1.4590000000000001}]},{"osd":4,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97199999999999998,"15min":0.97099999999999997},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.266},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.1000000000000001},"min":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.591,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.88}]},{"osd":6,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.0209999999999999,"15min":0.999},"min":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.1559999999999999,"15min":1.1639999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.726,"5min":1.764,"15min":1.764},"last":1.3759999999999999}]},{"osd":8,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":5.476,"15min":6.1980000000000004},"min":{"1min":0.77800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.641,"5min":104.761,"15min":104.761},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94999999999999996,"15min":0.93500000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":1.159}]},{"osd":10,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":16.478999999999999,"15min":19.009},"min":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.0899999999999999,"5min":363.67399999999998,"15min":363.67399999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3300000000000001,"15min":1.3360000000000001},"min":{"1min":0.91600000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.776,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.1279999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2390000000000001,"15min":1.234},"min":{"1min":0.78900000000000003,"5min":0.746,"15min":0.746},"max":{"1min":4.4649999999999999,"5min":4.4649999999999999,"15min":4.4649999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":2.633,"5min":1.6220000000000001,"15min":1.454},"min":{"1min":0.73199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":25.344999999999999,"5min":25.344999999999999,"15min":25.344999999999999},"last":1.484}]},{"osd":12,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.0820000000000001,"15min":1.056},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.431},{"interface":"front","average":{"1min":15.794,"5min":4.3339999999999996,"15min":2.4239999999999999},"min":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":289.64499999999998,"5min":289.64499999999998,"15min":289.64499999999998},"last":1.2929999999999999}]},{"osd":13,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3500000000000001,"15min":1.3400000000000001},"min":{"1min":0.51200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.2889999999999997,"5min":4.2889999999999997,"15min":4.2889999999999997},"last":1.405},{"interface":"front","average":{"1min":1.345,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.9370000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.044}]},{"osd":14,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.2889999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":1.083},{"interface":"front","average":{"1min":48.968000000000004,"5min":10.965,"15min":4.6310000000000002},"min":{"1min":0.874,"5min":0.753,"15min":0.753},"max":{"1min":950.322,"5min":950.322,"15min":950.322},"last":1.3180000000000001}]},{"osd":15,"last update":"Mon Sep 16 06:50:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.522,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.986,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.528},{"interface":"front","average":{"1min":1.4610000000000001,"5min":39.505000000000003,"15min":45.845999999999997},"min":{"1min":0.85799999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":3.9870000000000001,"5min":750.83299999999997,"15min":750.83299999999997},"last":1.2150000000000001}]}]},{"osd":10,"up_from":23,"seq":98784247845,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2379776,"data_stored":1396948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":50330050},"hb_peers":[2,3,4,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Sep 16 06:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.89800000000000002},{"interface":"front","average":{"1min":74.563000000000002,"5min":74.563000000000002,"15min":74.563000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":598.01499999999999,"5min":598.01499999999999,"15min":598.01499999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":29.286999999999999,"5min":6.4790000000000001,"15min":2.677},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":599.428,"5min":599.428,"15min":599.428},"last":1.492},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.86599999999999999,"15min":0.85199999999999998},"min":{"1min":0.51900000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.4470000000000001}]},{"osd":3,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.97699999999999998,"15min":0.97799999999999998},"min":{"1min":0.71199999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.101,"5min":87.593000000000004,"15min":102.008},"min":{"1min":0.748,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.401,"5min":981.82899999999995,"15min":981.82899999999995},"last":1.613}]},{"osd":4,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0429999999999999,"15min":1.0349999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.663},{"interface":"front","average":{"1min":1.004,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.395,"5min":1.516,"15min":1.516},"last":1.7170000000000001}]},{"osd":5,"last update":"Mon Sep 16 06:50:49 2024","interfaces":[{"interface":"back","average":{"1min":75.527000000000001,"5min":75.527000000000001,"15min":75.527000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":705.91800000000001,"5min":705.91800000000001,"15min":705.91800000000001},"last":2.278},{"interface":"front","average":{"1min":34.545999999999999,"5min":34.545999999999999,"15min":34.545999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":454.01900000000001,"5min":454.01900000000001,"15min":454.01900000000001},"last":2.121}]},{"osd":7,"last update":"Mon Sep 16 06:50:24 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1080000000000001,"15min":1.0960000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1899999999999999,"15min":1.194},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":2.016}]},{"osd":8,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.163,"15min":1.1759999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.458,"5min":2.089,"15min":2.089},"last":2.488},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2150000000000001,"15min":1.2270000000000001},"min":{"1min":0.88300000000000001,"5min":0.878,"15min":0.878},"max":{"1min":1.4490000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":2.1619999999999999}]},{"osd":9,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1990000000000001,"15min":1.2210000000000001},"min":{"1min":0.89200000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":2.3399999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":2.6019999999999999}]},{"osd":11,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":47.122999999999998,"5min":10.505000000000001,"15min":4.4020000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":501.79899999999998,"5min":501.79899999999998,"15min":501.79899999999998},"last":2.423},{"interface":"front","average":{"1min":3.294,"5min":1.7629999999999999,"15min":1.508},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":41.326000000000001,"5min":41.326000000000001,"15min":41.326000000000001},"last":2.4580000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":13,"last update":"Mon Sep 16 06:50:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":2.3769999999999998},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":2.5259999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":15,"last update":"Mon Sep 16 06:50:11 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.351,"15min":1.3520000000000001},"min":{"1min":1.0229999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6279999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":2.5600000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.034,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.9850000000000001,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":2.2400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T06:50:55.487 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T06:50:55.810 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T06:50:55.811 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T06:50:55.811 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T06:50:55.811 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:50:55.895 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T06:50:55.895 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T06:50:55.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:50:55.984 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T06:50:55.984 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-16T06:50:55.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:50:56.070 INFO:tasks.ceph.osd.8:Stopped 2024-09-16T06:50:56.070 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-09-16T06:50:56.070 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:50:56.166 INFO:tasks.ceph.osd.12:Stopped 2024-09-16T06:50:56.167 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T06:50:56.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:50:56.247 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T06:50:56.248 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T06:50:56.248 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:50:58.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-16T06:51:02.350 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T06:51:02.351 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-16T06:51:02.351 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.411 INFO:tasks.ceph.osd.9:Stopped 2024-09-16T06:51:02.412 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-09-16T06:51:02.412 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.482 INFO:tasks.ceph.osd.13:Stopped 2024-09-16T06:51:02.482 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T06:51:02.482 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.558 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T06:51:02.559 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T06:51:02.559 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.640 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T06:51:02.640 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-16T06:51:02.640 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.722 INFO:tasks.ceph.osd.10:Stopped 2024-09-16T06:51:02.723 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-09-16T06:51:02.723 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.802 INFO:tasks.ceph.osd.14:Stopped 2024-09-16T06:51:02.802 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T06:51:02.802 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.898 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T06:51:02.898 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T06:51:02.898 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:02.987 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T06:51:02.988 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-16T06:51:02.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:03.071 INFO:tasks.ceph.osd.11:Stopped 2024-09-16T06:51:03.071 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-09-16T06:51:03.072 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:03.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2024-09-16T06:51:03.686 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-16T06:51:09.172 INFO:tasks.ceph.osd.15:Stopped 2024-09-16T06:51:09.173 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T06:51:09.173 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T06:51:09.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:09.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~6s 2024-09-16T06:51:09.200 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T06:51:09.200 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T06:51:09.201 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:09.233 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T06:51:09.233 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T06:51:09.233 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T06:51:09.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:09.256 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T06:51:09.257 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T06:51:09.257 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:09.282 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T06:51:09.282 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T06:51:09.282 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T06:51:09.308 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T06:51:09.308 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T06:51:09.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-16T06:51:09.325 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:51:09.325 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T06:51:09.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:51:09.548 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T06:51:09.704 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:51:09.704 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-16T06:51:09.871 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:51:09.871 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-09-16T06:51:09.987 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:51:09.987 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T06:51:10.162 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:51:10.162 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T06:51:10.303 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:51:10.304 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-16T06:51:10.437 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:51:10.437 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-09-16T06:51:10.563 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:51:10.563 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T06:51:10.677 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:51:10.677 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T06:51:10.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:51:10.883 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-16T06:51:11.081 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:51:11.081 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-09-16T06:51:11.264 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:51:11.264 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T06:51:11.508 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:51:11.508 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T06:51:11.669 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:51:11.669 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-16T06:51:11.811 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:51:11.811 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-09-16T06:51:11.944 INFO:tasks.ceph:Archiving mon data... 2024-09-16T06:51:11.945 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/data/mon.a.tgz 2024-09-16T06:51:11.945 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-16T06:51:11.952 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.PuE2p5XRO8 2024-09-16T06:51:11.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PuE2p5XRO8 2024-09-16T06:51:12.101 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.PuE2p5XRO8 2024-09-16T06:51:12.166 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.PuE2p5XRO8 is 458KB 2024-09-16T06:51:12.228 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.PuE2p5XRO8 2024-09-16T06:51:12.234 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/data/mon.b.tgz 2024-09-16T06:51:12.234 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-16T06:51:12.240 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.gU3YtyiB7s 2024-09-16T06:51:12.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gU3YtyiB7s 2024-09-16T06:51:12.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.gU3YtyiB7s 2024-09-16T06:51:12.505 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.gU3YtyiB7s is 637KB 2024-09-16T06:51:12.567 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.gU3YtyiB7s 2024-09-16T06:51:12.572 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/data/mon.c.tgz 2024-09-16T06:51:12.572 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-16T06:51:12.578 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.iwvBELylhj 2024-09-16T06:51:12.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.iwvBELylhj 2024-09-16T06:51:12.770 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.iwvBELylhj 2024-09-16T06:51:12.835 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.iwvBELylhj is 637KB 2024-09-16T06:51:12.897 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.iwvBELylhj 2024-09-16T06:51:12.902 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T06:51:12.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T06:51:12.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T06:51:12.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T06:51:12.948 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T06:51:12.962 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T06:51:12.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T06:51:12.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T06:51:12.967 DEBUG:teuthology.orchestra.run.smithi115:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T06:51:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T06:51:12.989 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T06:51:13.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T06:51:13.023 INFO:teuthology.orchestra.run.smithi115.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T06:51:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T06:51:13.033 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T06:51:13.034 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi100/crash 2024-09-16T06:51:13.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T06:51:13.047 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi107/crash 2024-09-16T06:51:13.048 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T06:51:13.061 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi115/crash 2024-09-16T06:51:13.062 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T06:51:13.080 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi117/crash 2024-09-16T06:51:13.081 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T06:51:13.094 INFO:tasks.ceph:Compressing logs... 2024-09-16T06:51:13.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:13.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:13.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:13.129 DEBUG:teuthology.orchestra.run.smithi117:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:14.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2024-09-16T06:51:17.164 INFO:tasks.ceph:Archiving logs... 2024-09-16T06:51:17.165 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi100/log 2024-09-16T06:51:17.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T06:51:17.707 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi107/log 2024-09-16T06:51:17.708 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T06:51:18.145 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi115/log 2024-09-16T06:51:18.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T06:51:18.579 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi117/log 2024-09-16T06:51:18.580 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T06:51:18.985 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T06:51:18.996 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T06:51:18.997 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T06:51:18.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T06:51:19.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T06:51:19.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T06:51:19.119 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T06:51:19.128 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T06:51:19.128 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:51:19.131 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:51:19.133 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:51:19.136 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T06:51:19.142 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:51:19.142 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-09-16T06:51:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:+hv01.front.sepi 67.205.162.81 3 u 49 64 77 0.078 -3.592 2.106 2024-09-16T06:51:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:*hv02.front.sepi 63.231.80.2 3 u 54 64 77 0.075 -1.933 3.124 2024-09-16T06:51:19.143 INFO:teuthology.orchestra.run.smithi100.stdout:+hv03.front.sepi 74.6.168.72 3 u 52 64 77 0.068 -4.620 2.809 2024-09-16T06:51:19.143 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:51:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:51:19.144 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2024-09-16T06:51:19.145 INFO:teuthology.orchestra.run.smithi107.stdout:+hv01.front.sepi 67.205.162.81 3 u 56 64 77 0.049 -5.315 1.855 2024-09-16T06:51:19.145 INFO:teuthology.orchestra.run.smithi107.stdout:+hv02.front.sepi 63.231.80.2 3 u 58 64 77 0.091 +2.408 2.777 2024-09-16T06:51:19.145 INFO:teuthology.orchestra.run.smithi107.stdout:*hv03.front.sepi 74.6.168.72 3 u 55 64 77 0.080 -1.099 2.343 2024-09-16T06:51:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:51:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:51:19.146 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================== 2024-09-16T06:51:19.146 INFO:teuthology.orchestra.run.smithi115.stdout:+hv01.front.sepi 67.205.162.81 3 u 62 64 77 0.095 -1.332 0.860 2024-09-16T06:51:19.146 INFO:teuthology.orchestra.run.smithi115.stdout:+hv02.front.sepi 63.231.80.2 3 u 56 64 77 0.072 +2.515 0.652 2024-09-16T06:51:19.146 INFO:teuthology.orchestra.run.smithi115.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 77 0.090 +0.552 1.349 2024-09-16T06:51:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:51:19.149 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T06:51:19.149 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-09-16T06:51:19.149 INFO:teuthology.orchestra.run.smithi117.stdout:*hv01.front.sepi 67.205.162.81 3 u 55 64 77 0.113 -1.740 0.970 2024-09-16T06:51:19.149 INFO:teuthology.orchestra.run.smithi117.stdout:+hv02.front.sepi 63.231.80.2 3 u 50 64 77 0.074 +1.658 1.068 2024-09-16T06:51:19.149 INFO:teuthology.orchestra.run.smithi117.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 77 0.114 -0.059 1.276 2024-09-16T06:51:19.149 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T06:51:19.150 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T06:51:19.158 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T06:51:19.159 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T06:51:19.209 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T06:51:19.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T06:51:19.309 INFO:teuthology.task.internal:Duration was 851.643671 seconds 2024-09-16T06:51:19.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T06:51:19.360 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T06:51:19.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T06:51:19.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T06:51:19.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T06:51:19.368 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T06:51:19.405 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T06:51:19.405 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T06:51:19.405 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T06:51:19.447 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-16T06:51:19.447 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T06:51:19.458 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-16T06:51:19.458 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T06:51:19.468 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-16T06:51:19.468 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T06:51:19.478 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T06:51:19.478 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:19.491 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:19.502 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:19.511 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T06:51:19.552 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T06:51:19.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T06:51:19.563 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T06:51:19.570 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T06:51:19.579 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T06:51:19.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T06:51:19.667 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T06:51:19.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T06:51:19.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T06:51:19.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T06:51:19.703 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T06:51:19.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T06:51:19.727 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.734 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.737 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-16T06:51:19.743 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-16T06:51:19.748 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.757 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-16T06:51:19.771 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-16T06:51:19.837 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:51:19.842 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:51:19.885 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:51:19.890 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T06:51:19.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T06:51:19.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T06:51:19.904 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T06:51:19.905 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi100 2024-09-16T06:51:19.905 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T06:51:19.929 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi107 2024-09-16T06:51:19.929 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T06:51:19.950 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi115 2024-09-16T06:51:19.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T06:51:19.973 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906576/remote/smithi117 2024-09-16T06:51:19.974 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T06:51:19.996 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T06:51:19.997 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T06:51:19.999 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T06:51:20.002 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T06:51:20.018 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T06:51:20.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T06:51:20.049 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T06:51:20.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T06:51:20.076 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T06:51:20.076 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T06:51:20.079 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T06:51:20.081 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T06:51:20.082 INFO:teuthology.orchestra.run.smithi100.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 06:51 /home/ubuntu/cephtest 2024-09-16T06:51:20.083 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T06:51:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: 658635 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 06:51 /home/ubuntu/cephtest 2024-09-16T06:51:20.089 INFO:teuthology.orchestra.run.smithi115.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 06:51 /home/ubuntu/cephtest 2024-09-16T06:51:20.089 INFO:teuthology.orchestra.run.smithi117.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 06:51 /home/ubuntu/cephtest 2024-09-16T06:51:20.091 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T06:51:20.128 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T06:51:20.230 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 851.6436710357666 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T06:51:20.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T06:51:20.307 INFO:teuthology.run:pass